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SECTION  1 


INTRODUCTION  AND  SUMMARY 


This  study  seeks  criteria  for  evaluating  interdiction  operations 
opposing  the  flow  of  illegal  drugs  into  the  United  States.  Its  under¬ 
lying  purpose  is  to  find  analytically  sound  ways  to  assess  the  effec¬ 
tiveness  of  military  roles  in  antidrug  operations.  Most  such  roles  are 
in  direct  support  of  the  border  interdiction  campaign  where  law  enforce¬ 
ment  agencies  (LEAs)  combat  smugglers  through  searches,  seizures,  and 
arrests.  Because  military  forces  are  prohibited  from  conducting  these 
activities,  their  support  is  directed  at  enhancing  the  primary  interdic¬ 
tion  mission.  As  a  result,  the  bulk  of  this  study  concentrates  on 
finding  measures  of  effectiveness  (MOEs)  for  drug  interdiction  taken  as 
a  whole. 

The  results  of  this  study  are  addressed  to  three  groups: 

•  Persons  and  organizations  who  urge  LEAs  and  the  Department 
of  Defense  (DOD)  to  report  performance  of  the  antidrug 
mission  in  terms  of  MOEs 

•  Officials  in  these  front-line  agencies  who  are  struggling 
to  find  valid  MOEs 

•  Analysts  trying  to  solve  the  problem  for  the  other  two 
groups . 

Members  of  all  these  groups  should  find  the  study  a  source  of  concepts, 
methods,  and  results  that  will  affect  their  views  on  this  problem. 

SUMMARY 

Criteria  for  MOEs 

The  key  element  of  the  study's  evaluation  process  is  a  set  of 
11  attributes  that  are  ideal  properties  for  individual  MOEs,  plus  2  more 
attributes  that  apply  to  several  MOEs  used  in  combination.  Each  such 
attribute  is  accompanied  by  a  criterion  for  gauging  whether  the  MOE  is 
up  to  standard.  In  brief,  the  attributes  and  associated  criteria  are: 

•  Mission  oriented:  If  forces  act  to  optimize  their  MOE 
scores,  will  they  maximize  mission  accomplishment  as  well? 

•  Measurable:  Can  the  required  data  be  collected  reliably 
and  at  a  tolerable  cost? 

•  Quantitative:  Are  the  data  numeric  and  collected 
according  to  a  structured  procedure  that  gives  consistent 
results? 
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•  Discriminating:  Is  the  MOE's  statistical  uncertainty 
small  enough  to  recognize  real  performance  changes  over 
practical  time  intervals? 

•  Realistic:  Does  the  MOE  reflect  the  facts  of  the  opera¬ 
tional  situation,  not  some  remote  abstraction? 

•  Objective:  Given  common  methodology  and  data,  would  two 
different  people  compute  the  same  score  on  the  MOE  and 
come  to  a  common  interpretation  of  the  result? 

•  Appropriate:  Does  the  measured  performance  respond  pri¬ 
marily  to  activities  within  the  scope  and  authority  of  the 
forces  being  evaluated? 

•  Sensitive:  Do  MOE  movements  track  well  with  effects  the 
forces  actually  can  induce?  Can  external  factors  mask  the 
real  results? 

•  Promptly  responsive:  If  there  is  a  real  change  in  effec¬ 
tiveness,  how  long  is  the  delay  until  the  change  is 
detected  via  the  MOE? 

•  Simple  to  understand:  Can  people  at  all  levels  read  the 
documentation  and  correctly  explain  how  operations  relate 
to  the  MOE? 

•  Validated:  Have  field  tests  shown  that  the  MOE  is  practi¬ 
cal  to  implement  and  responds  as  intended  to  events? 

A  practical  system  likely  will  require  multiple  MOEs;  two 
additional  attributes  are  defined  for  assessing  suites  of  MOEs: 

•  Balanced:  Does  the  suite  of  MOEs  encompass  all  major 
mission  objectives?  Does  any  aspect  receive  undue 
prominence? 

•  Compact:  Is  the  suite  of  MOEs  as  small  as  possible  with¬ 
out  seriously  compromising  the  other  attributes  in  this 
list? 

Together,  these  13  attributes  and  associated  criteria  serve  as  a 
standard  for  evaluating  candidate  MOEs . 

MOEs  Evaluated 

The  study  evaluated  six  different  MOEs  under  this  standard. 

•  Raw  seizures:  tonnage  seized 
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•  Boarding  success  rate:  fraction  of  boarding  that  result 
in  a  seizure 

•  Interdiction  rate:  fraction  of  drug-smuggling  missions 
that  end  in  seizure 

•  Smuggler  success  rate:  fraction  of  all  drug  shipments 
reaching  the  U . S . 

•  Import-export  price  difference:  the  smuggler’s  markup 

•  Market  share:  fraction  of  national  imports  (for  a  given 
drug)  passing  through  a  given  area  of  responsibility 
(AOR) . 


RESULTS 

Evaluation  of  Interdiction  Forces  as  a  Whole 

The  six  MOEs  that  seem  attractive  on  the  surface  all  turn  out  to 
harbor  one  or  more  of  these  deficiencies  when  evaluating  real 
operations : 

•  Require  data  that  cannot  be  collected  in  real  operations 

•  Offer  incentives  to  conduct  operations  in  ways  that 
actually  undermine  the  mission 

•  Respond  strongly  to  events  that  have  nothing  to  do  with 
the  effectiveness  of  the  forces  being  evaluated 

•  Are  beset  by  such  low  event  rates  or  high  statistical 
variability  that  real  changes  in  performance  may  be 
recognized  only  when  the  MOE  moves  in  implausibly  large 
steps,  all  smaller  moves  being  indistinguishable  from  mere 
noise. 

These  same  MOEs,  however,  still  may  be  of  value  for  analysis, 
planning,  and  training  conducted  within  postulated  scenarios  where  the 
behavior  of  both  sides  can  be  examined  in  virtually  any  degree  of 
detail.  This  shift  of  venue  brings  a  corresponding  shift  of  emphasis 
from  the  problems  of  sample  statistics  to  these  of  realistically 
modeling  interdiction  forces,  smugglers,  and  their  interactions. 

Military  Roles  in  Interdiction 

Determining  measures  of  effectiveness  to  assess  the  contribution  of 
military  forces  to  the  drug  interdiction  effort  presents  additional 
problems.  First,  attempts  to  link  military  contributions  to  measures 
based  on  seizures  are  suspect  because  military  forces  lack  the  authority 
and  training  to  search  conveyances,  seize  property,  or  arrest 
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suspects;  only  law  enforcement  agencies  (Customs,  Coast  Guard,  Drug 
Enforcement  Administration,  state  and  local  police,  and  others)  may  do 
so.  Second,  it  is  all  but  impossible  to  separate  the  military 
contribution  from  the  functions  of  civilian  agencies  that  also  operate 
surveillance  assets,  communication  networks,  command  centers, 
interceptor  aircraft,  and  seagoing  vessels.  Virtually  everything  a 
military  unit  can  do  either  parallels  or  is  entangled  with  someone 
else's  performance. 

ORGANIZATION  OF  PAPER 

The  next  section  develops  the  family  of  attributes  that  character¬ 
ize  an  ideal  MOE  for  the  drug  interdiction  mission;  section  3  explores 
what  interdiction  means  in  the  antidrug  context;  sections  4  through  10 
evaluate  some  leading  candidates  for  drug  interdiction  MOEs  and  a  suite 
of  three  MOEs  in  combination;  section  11  summarizes  the  performance  of 
the  candidate  MOEs;  section  12  addresses  the  military  roles,  particular¬ 
ly  the  legal  restrictions  and  the  peculiar  problems  of  assessing  their 
contribution;  and  section  13  contains  the  conclusions  of  this  study. 
Appendix  A  explores  the  nature  of  MOEs- -what  they  are,  why  one  might 
want  some  MOEs,  and  the  overall  assessment  system  for  which  the  MOEs  are 
but  the  most  visible  element.  It  also  introduces  a  set  of  contrasts  to 
clarify  discussions  of  MOEs.  Appendix  B  contains  an  analysis  of 
boarding  success  rate  as  an  MOE,  and  appendix  C  explores  the  uses  of 
postulated  scenarios  (as  employed  in  exercises,  war  games,  and 
simulations) . 
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SECTION  2 


CRITERIA  FOR  ANTIDRUG  MEASURES  OF  EFFECTIVENESS 


As  a  prelude  to  evaluating  specific  MOEs  for  antidrug  operations, 
this  section  addresses  the  question  of  what  attributes  one  should  seek 
in  these  MOEs.  Eleven  such  attributes  are  proposed  for  evaluating 
individual  MOEs,  each  introduced  with  a  narrative  statement  followed  by 
a  specific  criterion  for  judging  whether  an  MOE  has  the  attribute.  Two 
additional  attributes  are  proposed  for  evaluating  suites  of  MOEs,  a 
reasonable  prospect  since  practical  systems  for  performance  assessment 
are  likely  to  include  multiple  MOEs.  This  list  extends  previous  efforts 
from  the  operations  research  literature  [1,  2,  and  3]  to  define 
desirable  qualities  of  MOEs;  as  far  as  could  be  determined  in  an 
extensive  survey  [4] ,  only  [3]  addresses  the  topic  for  antidrug 
operations. 

CRITERIA  FOR  INDIVIDUAL  MOEs 
Mission  Oriented 

MOEs  should  measure  how  well  operations  meet  mission  objectives. 
Stating  objectives  in  a  manner  suitable  for  measurement  can  be  the  most 
difficult  step  in  the  assessment  process.  One  must  take  care  to  capture 
the  substance  of  the  issue  in  the  MOE,  not  some  marginal  aspect  that 
happens  to  be  easy  to  measure.  The  chain  of  command  must  approach 
development  of  an  MOE  with  the  expectation  that  it  will  become  a 
shorthand  statement  of  the  mission.  CRITERION:  If  forces  act  to  attain 
a  top  score  on  the  MOE,  will  they  maximize  mission  accomplishment  as 
well? 

Measurable 

An  MOE  should  represent  a  measurable  concept,  the  required  data 
being  capable  of  routine  collection  with  reasonable  effort.  If  the  data 
prove  measurable  in  theory  but  not  in  practice,  it  may  be  possible  to 
find  some  closely  correlated  surrogates,  albeit  at  the  price  of  addi¬ 
tional  uncertainty  when  interpreting  the  results.  CRITERION:  Can  the 
required  data  be  collected  reliably  and  at  a  tolerable  cost? 

Quantitative 

Numerical  data  and  MOEs  offer  important  advantages  over  qualitative 
alternatives.  Numbers  facilitate  powerful  mathematical  tools  and  tend 
to  reduce  the  chances  of  human  judgement  becoming  embedded  in  the 
system.  Still,  the  data-collection  system  needs  safeguards  to  assure 
that  even  numerical  reports  are  the  product  of  a  defined  measurement 
process,  not  merely  someone's  intuition;  it  would  be  a  poor  system  if 
the  output  changed  significantly  as  a  result  of  turnover  in 
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leadership  or  a  key  staff  position.  CRITERION:  Are  the  data  numeric 
and  collected  according  to  a  structured  procedure  that  gives  consistent 
results? 

Discriminating 

MOEs  should  exploit  sharp  distinctions  to  expose  differences  among 
alternatives  and  detect  real  changes  in  performance  over  time.  Even 
carefully  drawn  theoretical  contrasts  can  become  fuzzy  when  faced  with 
the  errors,  omissions,  and  ambiguities  of  data  collected  from  opera¬ 
tions.  As  a  result,  the  reported  value  of  an  MOE  must  be  recognized  as 
only  an  estimate  of  actual  performance,  deserving  to  be  accompanied  by  a 
statistical  statement  of  uncertainty.  One  way  to  narrow  the  uncertain¬ 
ty  is  to  accumulate  more  data,  usually  through  lengthening  the  time 
interval  of  the  assessment;  for  example,  year-to-year  changes  in  per¬ 
formance  are  detectable  with  greater  confidence  than  month- to -month 
changes.  The  statement  of  uncertainty  for  an  estimate  should  be  consid¬ 
ered  as  important  as  the  estimate  itself.  CRITERION:  Is  the  MOE's 
statistical  uncertainty  small  enough  to  recognize  real  performance 
changes  over  practical  time  intervals? 

Realistic 

One  goal  of  the  MOE  design  process  should  be  to  capture  the  essence 
of  the  operation  while  making  due  allowance  for  real-world  limitations. 
It  is  easy  to  fall  into  the  trap  of  reshaping  the  assessment  problem 
into  one  with  a  convenient  solution,  while,  in  the  process,  letting  the 
substance  of  the  question  slip  away.  CRITERION:  Does  the  MOE  reflect 
the  facts  of  the  operational  situation,  not  some  remote  abstraction? 


1.  Suppose  some  operation  achieves  an  MOE  score  that  is  larger  than  the 
norm.  How  can  an  official  decide  whether  this  apparent  improvement  is  a 
real  gain  over  the  past?  The  answer  depends  on  both  the  uncertainty  in 
the  historic  value  and  the  size  of  the  sample  used  in  computing  the  new 
score.  With  an  appropriate  assumption  for  the  form  of  the  probability 
distribution  describing  such  scores,  a  statistician  can  compute  the 
probability  that  a  new  score  this  large  is  consistent  with  the  historic 
distribution.  This  probability  is  a  direct  estimate  of  the  risk  that 
the  official  will  be  wrong  if  he  decides  that  the  new  score  is  different 
from  the  historic  level.  If  the  risk  is  unacceptable,  he  concludes  that 
the  scores  are  really  the  same,  any  apparent  difference  merely 
reflecting  statistical  uncertainty  in  the  estimates. 


2-2 


Objective 


The  results  of  the  assessment  system  should  have  as  little  to  do 
with  institutional  biases  and  expectations  as  possible.  More  general¬ 
ly,  objectivity  is  enhanced  if  scientific  analysis  has  established  the 
nature  and  strength  of  the  relationship  between  the  MOE  and  the  activity 
it  purports  to  measure.  CRITERION:  Given  common  methodology  and  data, 
would  two  different  people  compute  the  same  score  on  the  MOE  and  come  to 
a  common  interpretation  of  the  result? 

Appropriate 

MOEs  should  be  keyed  to  the  level  of  command  and  scope  of  operat¬ 
ions  under  appraisal.  It  does  little  good  to  evaluate  a  commander  on 
events  beyond  his  influence.  Though  the  nominal  mission  may  be  the  same 
at  several  different  echelons,  each  level  might  have  its  own  distinctive 
suite  of  MOEs,  some  for  external  reporting  and  others  for  internal 
management.  CRITERION:  Does  the  measured  performance  respond  primarily 
to  activities  within  the  scope  and  authority  of  the  forces  under 
evaluation? 

Sensitive 

A  sound  MOE  should  respond  to  changes  in  the  quantity,  quality,  and 
character  of  operational  events;  if  the  answer  stays  about  the  same 
regardless  of  how  tactical  results  change,  the  MOE  is  not  very  illumi¬ 
nating.  By  the  same  token,  the  MOE  should  be  slow  to  react  to  extrane¬ 
ous  factors;  an  MOE  that  suffers  wide  swings  in  response  to  the  actions 
of  other  parties  will  tend  to  obscure  rather  than  illuminate  the  mission 
performance  of  the  forces  under  scrutiny.  CRITERION:  Do  MOE  movements 
track  well  with  effects  the  forces  actually  can  induce?  Can  external 
factors  mask  the  real  results? 

Promptly  Responsive 

The  process  of  collecting,  correcting,  and  analyzing  data 
necessarily  takes  time,  but  delays  grow  as  the  scheme  adds  reporting 
sources,  especially  ones  outside  the  chain  of  command.  At  the  same 
time,  it  is  self-defeating  to  rush  the  process  for  an  answer  and  then 
retract  it  later  in  the  face  of  more  definitive  results.  An  MOE  that 
produces  month- to -month  comparisons  is  far  more  responsive  than  another 
that  can  support  only  year-to-year  contrasts;  the  time  scale  might  be 
dictated  by  either  the  pace  of  interdiction  events  or  the  availability 
of  essential  data.  CRITERION:  If  there  is  a  real  change  in 


1.  Apart  from  the  MOEs  and  their  supporting  data  stream,  the  commander 
and  his  staff  always  retain  full  authority  to  comment  on  operations  and 
bring  out  issues  beyond  the  scope  of  any  formal  assessment  system. 
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effectiveness,  how  long  is  the  delay  until  the  change  is  detected  via 
the  MOE?  How  does  the  time  scale  compare  with  the  typical  action- 
reaction  cycle  of  the  effects  the  MOE  is  designed  to  measure? 

Simple  To  Understand 

If  an  MOE  is  to  perform  well,  it  must  be  readily  understandable  to 
all  persons  directly  involved  with  collecting  data,  compiling  reports, 
and  interpreting  results .  Authoritative ,  readable ,  and  concise  documen¬ 
tation  is  central  to  acquiring  this  understanding.  Comparison  and 
aggregation  of  results  across  a  range  of  activities  depends  on  all 
levels  of  command  sharing  a  common  perspective  on  the  meaning  of  the 
MOEs  and  their  supporting  data.  CRITERION:  Can  people  at  all  levels 
read  the  documentation  and  correctly  explain  how  operations  relate  to 
the  MOE?  (Note  that  the  criterion  puts  the  burden  on  concept  formula¬ 
tion  and  documentation,  not  on  the  readers.) 

Validated 

Any  serious  plan  for  assessing  performance  in  the  antidrug  mission 
needs  a  thorough  development  and  test  phase;  otherwise,  fundamental 
shortcomings  might  appear  during  operations  and  undermine  the  credibili¬ 
ty  of  the  system.  Whatever  concepts  finally  emerge  from  this  process, 
they  will  need  to  undergo  testing  to  demonstrate  that  their  theoretical 
properties  actually  work  well  enough  in  practice  to  justify  the  price  of 
adoption.  CRITERION:  Have  field  tests  shown  that  the  MOE  is  practical 
to  implement  and  responds  as  intended  to  events?  In  particular,  has  the 
validation  process  demonstrated  that  the  essential  data  exhibit  the 
required  characteristics  when  collected  via  methods  planned  for  routine 
use?  If  field  tests  require  large  start-up  costs,  Monte  Carlo  simula¬ 
tion  methods  might  be  an  economical  way  to  search  for  fatal  flaws ;  a 
final  endorsement  still  should  rest  on  a  full-fledged  encounter  with  the 
real  world. 

COLLECTIVE  CRITERIA  FOR  MOE  SUITES 

Capturing  all  important  aspects  of  drug  interdiction  operations 
almost  surely  is  beyond  the  scope  of  any  single  measure;  success  proba¬ 
bly  will  require  a  suite  of  mutually  supporting  MOEs  to  encompass  the 
various  missions  and  operations.  Though  most  of  the  criteria  just 
described  can  be  used  to  evaluate  a  single  MOE,  the  evaluation  process 
probably  is  best  performed  in  the  context  of  a  full  system  of  MOEs  to 
reveal  how  they  complement,  duplicate,  and  conflict  with  each  other. 

Balanced 

The  suite  of  MOEs  should  cover  all  important  aspects  of  the  mis¬ 
sion.  Some  elements  will  prove  easier  to  tap  than  others,  but  without  a 
determined  effort  to  achieve  balance,  the  suite  of  MOEs  will  carry  an 
implicit  weighting  of  mission  objectives  that  will  skew  operations. 
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CRITERION:  Does  the  suite  of  MOEs  encompass  all  major  mission 
objectives?  Does  any  aspect  receive  undue  prominence? 

Compact 

Ideally,  each  member  of  a  suite  of  MOEs  should  tap  its  own  particu¬ 
lar  aspect  of  the  problem  while  the  other  MOEs  remain  largely  unrespon¬ 
sive  to  this  same  aspect.  More  technically,  the  system  designer  should 
seek  MOEs  that  tend  to  be  mutually  independent,  a  property  that  supports 
spanning  the  decision  space  with  a  minimum  number  of  MOEs.  A  small 
suite  of  MOEs  reflecting  relatively  pure  effects  is  preferred  to  long 
and  complicated  lists  of  every  conceivable  MOE.  Some  duplication  and 
overlap  are  inevitable,  and  some  redundancy  may  even  be  introduced 
deliberately  if  confirmation  is  needed  on  critical  issues.  CRITERION: 

Is  the  suite  of  MOEs  as  small  as  possible  without  seriously  compromising 
the  other  attributes  in  this  list? 

CONCLUSION 

This  list  of  selection  criteria  is  offered  as  a  guide  to  MOE  devel¬ 
opment  and  evaluation.  In  all  likelihood,  no  suite  of  MOEs,  or  even 
individual  measures,  could  come  out  perfect  on  every  criterion.  On  the 
other  hand,  a  candidate  MOE  that  fails  totally  on  any  one  of  these 
criteria  deserves  a  hard  look  as  to  whether  it  has  any  utility.  The 
important  point  is  to  consider  these  qualities  in  the  design  stage  of 
the  assessment  system  and  identify  how  the  shortfalls  will  constrain 
interpretations  of  results  from  real  operations  or  postulated  scenarios. 
Sections  4  through  9  explore  six  candidate  MOEs  with  particular  atten¬ 
tion  to  their  most  prominent  features ,  including  a  systematic  assessment 
of  performance  on  these  criteria.  Section  10  applies  these  same  criter¬ 
ia  to  a  suite  of  three  mutually  supporting  MOEs . 


2-5 


SECTION  3 


THE  INTERDICTION  MISSION 


The  immediate  effects  of  interdiction  fall  on  smugglers  as  seizures 
and  arrests,  penalties  that  only  LEAs  may  impose.  Military  forces  serve 
in  the  national  war  on  drugs  primarily  by  providing  surveillance,  com¬ 
munications,  logistics,  and  other  support  to  enhance  effectiveness  of 
the  LEAs.  Arranging  the  roles  this  way  means  that  the  value  of  such 
military  support  is  indeterminate  unless  there  is  a  clear  understanding 
of  the  value  of  interdiction.  Accordingly,  this  section  explores  what 
interdiction  means  in  the  antidrug  context. 

THE  PLACE  OF  INTERDICTION  IN  THE  WAR  ON  DRUGS 

Interdiction  focuses  on  smuggling  methods  that  bypass  points  of 
entry  (POEs)  and  evade  the  formal  customs  clearance  process.  Smugging 
conveyances  include:  fishing  vessels,  pleasure  craft,  small  merchant 
ships,  private  aircraft,  motor  vehicles  operated  off-road,  even  pack 
animals  and  human  bearers.  The  principal  participants  in  interdiction 
operations  include  the  Customs  Service,  Coast  Guard,  Border  Patrol,  and 
military  services. 

In  contrast,  legitimate  travelers  and  cargo  entering  the  United 
States  present  their  papers  and  clear  customs  at  POEs.  Only  a  small 
percentage  of  the  traffic  passing  through  these  points  can  receive  close 
examination,  posing  a  huge  potential  for  illegal  drug  imports.  The  flow 
includes  large  merchant  vessels,  cruise  ships,  cargo  containers,  trains, 
and  commercial  airliners.  Smugglers  exploit  cargo,  luggage,  and  the 
bodies  of  travelers  in  a  wide  variety  of  ways  to  conceal  illegal 
drugs.  The  Immigration  and  Naturalization  Service  and  the  Customs 
Service  are  the  principal  agencies  stopping  drugs  at  POEs;  however. 
National  Guard  troops  and  military  working  dog  teams  have  growing  roles 
in  cargo  searches  (conducted  out  of  the  public  eye) . 

Interdiction  at  and  beyond  the  borders  directly  complements  inspec¬ 
tions  at  POEs;  drug  smugglers  (as  a  group  if  not  individually)  can  shift 
the  weight  of  their  effort  from  one  to  the  other  based  on  perceptions  of 
costs  and  peril.  Accordingly,  the  overall  antidrug  campaign  is  best 
served  if  these  two  segments  are  coordinated  to  deny  smugglers  any  low- 
risk  options. 

LEAs  typically  have  different  points  of  emphasis  about  interdiction 
than  do  military  forces;  they  focus  on  investigative  activity  where 
arrests  bring  the  accused  into  custody  and  seizures  provide  the  evidence 
necessary  to  clinch  the  case.  As  a  result,  interdiction  must  be 
executed  with  full  regard  for  due  process ,  protection  for  the  rights  of 
the  accused,  and  preservation  of  an  unbroken  chain  of  custody  for  the 
evidence.  Error  in  any  of  these  respects  can  damage  or  negate  the 
prosecution's  case,  leaving  only  the  drugs  and  some  seized  property  as 
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reward  for  the  effort;  smuggling  organizations  have  proven  willing  to 
take  such  losses  in  stride,  writing  them  off  as  a  cost  of  doing 
business . 

OFFICIAL  GOALS  OF  INTERDICTION 

The  National  Drug  Control  Strategy  [5]  states  the  goals  of  inter¬ 
diction  in  the  following  terms: 

The  ultimate  goal  of  interdiction  is  to  deter  drug 
smuggling  by  intercepting  and  seizing  illicit  drug 
shipments  entering  the  United  States.  This 
disruption  of  drug  trafficking  operations  raises  the 
traffickers'  cost  of  doing  business  by  forcing  them 
to  take  expensive  countermeasures:  using  longer  and 
more  circuitous  routes;  training  new  personnel  to 
replace  those  apprehended;  purchasing  sophisticated 
electronic  equipment  to  detect  law  enforcement 
surveillance;  developing  new  concealment  techniques;' 
replacing  expensive  seized  assets;  and  stockpiling 
drugs  closer  to  the  production  area,  thus  making  them 
more  vulnerable  to  foreign  law  enforcement  efforts. 

(p.  93) 

The  principal  goal  of  air  interdiction  operations  is 
to  deter  smugglers  using  aircraft  by  denying  them 
safe,  direct,  and  economical  routes  to  major 
distribution  areas  in  this  country.  A  related  goal 
is  to  seize  the  drugs,  the  aircraft,  and  the 
smugglers,  (p.  95) 

Our  maritime  interdiction  efforts  focus  principally 
on  deterring  drug  smuggling  by  monitoring  seaborne 
smuggling  routes,  detecting  and  seizing  drug¬ 
smuggling  vessels,  and  arresting  their  crews. 

(p.  97) 

Thus,  the  official  goal  of  interdiction  is  to  deter  drug  smuggling;  the 
mechanisms  of  enforcement  include  denial  of  preferred  routes,  seizures 
of  drugs  and  related  assets,  and  arrests  of  smugglers.  Taken  to 
extreme,  fully  effective  interdiction  might  become  so  certain  and  the 
consequences  so  costly  that  only  daredevils  would  run  the  risk. 

Intensified  interdiction  is  supposed  to  result  in  reduced 
smuggling,  all  other  things  being  equal,  but,  in  the  decade  of  the 
eighties,  each  year  has  seen  increases  in  both  the  resources  devoted  to 
interdiction  and  the  quantities  of  drugs  sold  on  the  retail  market. 
Smugglers  have  compiled  a  most  respectable  record  of  adaptation  and 
evasion;  when  one  avenue  becomes  difficult,  they  open  another. 
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Incremental  increases  in  law  enforcement  effort  continue  to  provide  the 
stimulus  to  find  new  opportunities ,  but  not  yet  the  motivation  for  a 
mass  exit  from  the  trade. 

A  fundamental  problem  in  this  mission  statement  is  that  it  fails  to 
recognize  that  deterrence  is  the  product  of  much  more  than  interdiction 
alone.  Deterrence  implies  giving  players  in  the  drug  trade  convincing 
reasons  to  halt,  restrain,  or  redirect  their  activity.  Personal  risks 
are  increased  by  raising  the  probabilities  of  arrest,  prosecution,  and 
jail  for  all  drug-related  activities.  Rewards  are  diminished  by  inter¬ 
rupting  the  flow  of  profits  and  other  returns;  specific  effects  include 
increased  costs,  an  eroded  stream  of  revenues,  undermined  predictability 
of  revenues,  and  blocked  paths  for  repatriation  of  profits.  Clearly, 
interdiction  taken  alone  has  a  limited  capacity  to  deter  drug 
trafficking;  it  affects  just  part  of  the  transportation  complex,  but 
nothing  of  production,  distribution,  or  money  laundering. 

Deterrence  has  not  been  translated  into  a  set  of  goals  with  opera¬ 
tional  meaning  for  interdiction  forces.  Maybe  it  cannot  be  translated. 
What  then  are  interdiction  forces  doing?  If  they  justify  their  activi¬ 
ties  in  terms  of  deterrence,  is  the  rationale  valid?  More  often,  the 
agencies  do  not  bother  to  speak  of  deterrence  at  all  and  so  must  be 
presumed  to  be  working  against  some  other  goal.  In  either  case,  the 
disconnect  in  the  chain  of  leadership  leaves  no  explicit  link  between 
actions  and  goals . 

RELATING  INTERDICTION  TO  NATIONAL  LEVEL  MOEs 

The  General  Accounting  Office  cites  macro-scale  criteria  to  drama¬ 
tize  the  size  and  intractable  nature  of  the  illegal  drug  problem  [6]: 

•  Street  prices  continue  in  a  long-run  decline. 

•  Retail  purity  is  increasing. 

•  Estimates  of  quantities  imported  and  sold  to  users 
continue  to  climb. 

•  Emergency  room  case  loads  for  illegal  drug  use  keep 
growing . 

•  Numbers  of  drug  users  are  larger  than  a  few  years  ago, 
though  recent  survey  results  show  small  declines. 

Such  price-purity-availability  criteria  have  a  place  in  assessing 
whether  the  national  campaign  against  illegal  drugs  is  making  headway; 
most  of  them  are  included  in  the  quantitative  goals  set  in  [5] .  There 
is  a  serious  problem,  however,  with  using  these  same  criteria  to 
evaluate  the  performance  of  any  single  agency  or  program  (as  in  [7]). 

For  interdiction  in  particular,  such  macro -measures  are  several  steps 
removed  from  what  the  interdiction  forces  actually  can  achieve: 
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intercept  some  fraction  of  shipments  on  smuggling  vehicles . 

Quantitative  valuations  of  these  macro -measures  are  strongly  affected  by 
events  beyond  the  reach  of  interdiction  forces: 

•  Domestic  production:  Reputedly,  the  world's  best 
marijuana  grows  in  the  U.S.,  satisfying  a  large  fraction 
of  the  national  market;  should  crystal  methamphetamine 
("ice")  grow  from  footholds  in  the  western  U.S.  to  achieve 
national  popularity,  domestic  production  in  illicit  chem¬ 
istry  labs  could  outstrip  imports. 

•  Buffer  stocks  in  the  U.S.:  Cocaine  in  stateside  storage 
is  often  said  to  approximate  six  months  of  current  demand. 

•  Smuggling  via  concealment  in  commercial  shipments,  notably 
the  maritime  container  trade. 

•  Smuggling  by  courier,  a  key  method  for  importing  heroin 
and  other  opiates. 

These  same  macro -measures  also  respond  to  market  factors  having 
little  or  nothing  to  do  with  interdiction.  Dealers  at  all  levels 
compete  for  market  share,  at  times  engaging  in  wars,  price  and  other¬ 
wise;  others  who  enjoy  a  locally  dominant  position  in  the  market  can 
gouge  their  customers,  at  least  for  a  while;  and  the  entire  market 
responds  to  changing  consumer  fashion- -this  year's  drug  of  choice  may  be 
overtaken  by  a  new  fad.  Patterns  of  preference  can  vary  widely  by 
location  as  well  as  time;  just  two  of  many  pertinent  examples  are  the 
popularity  of  phencyclidine  (PCP)  in  Washington,  DC,  and  the  slow  accep¬ 
tance  of  crack  cocaine  in  Chicago. 

Furthermore,  interdiction  is  only  one  facet  of  a  much  larger 
effort,  virtually  all  of  whose  elements  can  affect  these  same  macro¬ 
scale  measures : 

•  Domestic  law  enforcement  programs  (investigation,  prosecu¬ 
tion,  imprisonment,  rehabilitation) 

•  Education  programs  to  discourage  initial  or  continued  use 

•  Treatment  programs  for  addicts 

•  Antidrug  programs  of  other  countries  and  international 
organizations . 

The  credit  (or  blame)  for  changes  in  such  price-purity-availability 
measures  would  be  extraordinarily  difficult  to  allocate  among  all  these 
factors.  On  a  purely  technical  level,  it  might  be  possible  to  design  an 
input-output  matrix,  but  with  little  hope  of  collecting  the  data  neces¬ 
sary  for  credibly  estimating  its  coefficients.  As  a  result,  the 
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subsequent  sections  of  this  paper  explore  measures  much  more  closely 
related  to  the  direct  effects  of  interdiction. 


SOME  LARGE-SCALE  EFFECTS  OF  INTERDICTION 

Interdiction  also  has  large-scale  effects  on  relationships  within 
the  smuggling  community,  some  of  which  are  not  obvious  to  the  casual 
observer  but  should  not  be  overlooked  while  trying  to  evaluate  interdic¬ 
tion  operations.  Otherwise,  the  outcomes  may  be  unanticipated,  and 
perhaps  unintended. 

Drug  smuggling  involves  large  numbers  of  individuals  and  organiza¬ 
tions  whose  resources,  skills,  sophistication,  experience,  and  other 
attributes  vary  over  a  wide  range.  Unless  interdiction  is  directed 
specifically  at  those  positioned  near  the  high  end  of  the  spectrum,  the 
burden  of  seizures  and  arrests  naturally  falls  most  heavily  on  those  at 
the  low  end.  To  the  extent  that  the  interdiction  campaign  is  blind  to 
these  distinctions  (no  special  priority  goes  to  one  smuggler  over 
another- -they  are  all  bad),  the  more  organized  and  ruthless  players 
actually  stand  to  improve  their  relative  positions  as  a  result  of  inter¬ 
diction;  it  is  safe  to  say  that  strengthening  the  most  notorious  drug 
smugglers  should  not  be  an  objective  of  U.S.  government  policy. 

This  anomalous  situation  follows  directly  from  the  fact  that 
novices  are  far  more  likely  to  fall  prey  to  interdiction  than 
established  smugglers.  The  interdiction  program  raises  barriers  to  new 
entrants;  as  interdiction  thins  the  ranks  at  the  low  end  of  the  experi¬ 
ence  spectrum,  the  remaining  smugglers  enjoy  growing  market  share.  Cost 
increases  induced  by  interdiction  are  passed  through  to  the  customer  as 
higher  import  prices  than  would  pertain  otherwise,  but  the  established 
smuggler  realizes  the  higher  prices  without  bearing  the  full  burden  of 
cost  increases,  perhaps  even  enjoying  enlarged  profit  margins. 

Another  effect  of  higher  prices  is  marginal  growth  in  the  propen¬ 
sity  for  fraud,  theft,  and  violence  among  smugglers  because  they  are 
dealing  in  an  increasingly  valuable  product.  That  result  may  not  seem 
so  bad  if  the  smugglers  are  the  ones  who  suffer,  but  smugglers  vary  in 
their  tendency  toward  violence  and  ruthlessness;  those  who  are  most 
inclined  are  most  rewarded.  This  effect  spills  over  into  society  as 
further  assaults  against  morality,  respect  for  law  and  order,  and  peace 
in  the  streets. 

The  pressure  from  successful  interdiction  encourages  smugglers  to 
search  for  adaptations  and  alternatives ,  some  of  which  might  turn  out  to 
be  clear  improvements  over  their  old  methods.  For  example,  it  remains 


1.  See  [8]  for  a  readable  microeconomic  analysis  covering  these  points 
and  other  aspects  of  drug  law  enforcement. 

2.  Prohibition  produced  exactly  the  same  result;  organized  crime  drove 
small  operators  out  of  business  within  a  few  years  [9]. 
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to  be  seen  whether  the  development  of  large-scale  drug  routes  through 
Mexico  in  the  face  of  interdiction  in  the  Caribbean  amounts  to  a  net 
gain  for  the  government  or  the  smugglers. 

SUMMARY 

The  national  strategy  declares  the  principal  purpose  of  interdic¬ 
tion  is  deterrence  against  smuggling;  denial  of  routes,  seizures  of 
drugs,  and  arrests  of  smugglers  are  not  the  ultimate  goals,  merely  means 
to  the  end.  Upon  closer  inspection,  however,  deterrence  appears  as  an 
abstraction  that  cannot  be  linked  to  interdiction  operations  in  any 
practical  terms .  Nor  can  interdiction  bear  the  full  burden  for 
achieving  deterrence;  most  other  aspects  of  the  war  on  drugs  contribute 
as  well.  As  a  result,  there  seems  no  obvious  way  to  evaluate  interdic¬ 
tion  directly  in  terms  of  deterrence,  and  no  other  statement  of  the 
mission  has  official  blessing.  Absent  an  operationally  meaningful 
mission  to  guide  evaluation,  the  MOEs  considered  in  subsequent  sections 
concentrate  on  the  immediate  effects  that  are  primary  results  of 
interdiction. 
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SECTION  4 


RAW  SEIZURES 


Many  statistics  about  drug  smuggling  are  beset  with  crippling 
anomalies  and  uncertainties;  numbers  reported  to  four  significant  digits 
can  later  prove  to  be  in  error  by  an  order  of  magnitude.  Outstanding 
exceptions  to  this  principle  are  statistics  on  drug  seizures  and 
smuggler  arrests ;  LEAs  have  succeeded  in  counting  arrests  and  measuring 
seizures  with  high  accuracy. 

INTERPRETATION  OF  RAW  SEIZURE  STATISTICS 

Seizures  and  arrests  traditionally  have  been  cited  as  evidence  of 
success  in  the  interdiction  mission.  It  is  hard  to  find  a  summary  of 
antidrug  activity  that  fails  to  give  prominence  to  seizures  whenever 
some  claim  of  credit  might  be  justifiable.  On  the  surface,  a  pattern  of 
rising  numbers  seems  plausible  as  an  indicator  of  growing  impact  on 
smugglers  (assumes  seizures  are  growing  faster  than  quantities 
smuggled).  Paradoxically,  if  the  pattern  reverses  to  falling  numbers  of 
seizures  and  arrests ,  boosters  for  interdiction  still  claim  success , 
this  time  insisting  that  the  decline  shows  deterrence  is  working 
(assumes  quantities  smuggled  are  falling  too,  ideally  at  least  as  fast 
as  seizures) .  Skeptics  interpret  the  numbers  in  a  directly  opposite 
manner:  a  rising  number  of  seizures  is  cited  as  evidence  of  increased 
smuggling  (assumes  interdiction  forces  continue  seizing  about  the  his¬ 
toric  fraction  of  smuggled  drugs) ;  a  falling  series  is  said  to  show  that 


1.  Few  examples  are  as  dramatic  as  the  recent  shift  in  estimates  of 
Mexican  marijuana  production,  but  the  apparent  disregard  for  statistical 
uncertainty  is  typical  of  official  statistics  on  illegal  drugs .  For 
1988,  the  State  Department  [10]  reported  production  of  Mexican  marijuana 
at  5,655  metric  tons  and  projected  4,750  metric  tons  for  1989.  The  next 
comparable  report  [11]  estimated  1989  production  at  47,590  metric  tons, 
ten  times  as  much.  The  difference  is  attributed  not  to  a  major  increase 
in  cultivation  but  to  a  change  in  estimation  methodology;  figures  for 
previous  years  remained  unaltered,  though  they  were  conceded  to  be 
probable  underestimates.  Despite  having  shown  that  alternative  methods 
can  produce  answers  differing  by  an  order  of  magnitude,  the  State 
Department  continues  to  report  its  figures  as  though  they  are  precise, 
not  even  hinting  that  some  range  of  uncertainty  goes  with  the  estimate. 

2.  The  El  Paso  Intelligence  Center  (EPIC)  keeps  statistics  for  all  the 
federal  LEAs  and  publishes  summary  results  in  an  annual  report  [12]. 
Before  EPIC  became  the  designated  scorekeeper,  however,  national  figures 
were  derived  by  summing  statistics  compiled  from  each  agency;  the 
numbers  were  substantially  inflated  because  of  extensive  double  counts 
as  drug  busts  frequently  involved  more  than  one  agency,  each  including 
all  relevant  events  in  its  figures.  Coast  Guard  seizures  (and  most  Navy 
involvement)  are  detailed  twice  per  year  in  [13]. 
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the  smugglers  have  become  clever  enough  to  evade  interdiction  (assumes 
the  amounts  smuggled  have  remained  steady,  or  at  least  not  fallen  as 
fast  as  seizures). 

The  embedded  assumptions  by  both  sides  reflect  not  only  hidden 
agendas  but  also  the  difficulty  of  making  any  sense  from  raw  seizure 
statistics  without  referring  to  associated  levels  of  smuggling  and 
interdiction  activity;  standing  alone,  seizures  are  simply  numbers 
without  a  context.  As  a  result,  raw  seizures  as  an  MOE  is  a  measure  of 
neither  effectiveness  nor  efficiency.  Although  the  results  are  evidence 
of  some  tactical  success,  every  observer  has  to  figure  out  for  himself 
what  the  numbers  might  mean. 

RAW  SEIZURES  EVALUATED  ON  MOE  CRITERIA 

Mission  Oriented 

Probably  not.  Seizures  contribute  to  performance  of  the  antidrug 
mission- -drugs  and  smugglers  are  taken  out  of  circulation;  other 
smugglers  are  reminded  that  there  are  real  risks;  and  the  costs  of  doing 
business  are  nudged  up- -but  a  determined  effort  to  maximize  seizures 
holds  serious  potential  for  unintended  effects.  First,  interdiction 
forces  might  validly  maximize  seizures  by  concentrating  on  the  least 
experienced  smugglers  and  largely  ignoring  the  more  sophisticated, 
better-equipped  veterans.  Such  a  strategy  might  look  good  on  the 
surface  as  the  numbers  are  reported,  but  it  avoids  the  toughest  part  of 
the  problem.  Second,  smugglers  appreciate  that  they  face  risks  and 
expect  some  fraction  of  their  shipments  will  be  seized;  the  rational 
step  for  them  is  to  make  those  seizures  serve  their  purposes  as  much  as 
possible.  Smugglers  have  been  known  to  use  deliberate  sacrifices  to 
give  interdiction  forces  a  score,  but  simultaneously  exploited  the 
distraction  to  push  through  a  major  shipment;  the  effect  is  something 
like  a  burglar  who  brings  along  a  meaty  bone  to  occupy  the  watchdog 
while  the  master's  goods  go  out  the  door.  Third,  if  police  officials 
are  susceptible  to  corruption  (a  common  factor  overseas  and  an 
occasional  problem  even  in  the  U.S.),  sham  seizures  might  be  staged  as  a 
cooperative  event  to  make  the  police  look  good  while  the  main  trade  goes 
on  unimpeded. 

Measurable 

Yes.  Measurability  is  the  strongest  feature  of  seizures  as  an  MOE; 
numbers  of  interdiction  events  and  the  quantities  of  contraband  seized 
are  reported  in  detail.  Years  ago,  significant  errors  arose  from 
multiple  reporting  as  every  agency  involved  in  a  seizure  staked  its 
claim,  but,  now,  the  El  Paso  Intelligence  Center  keeps  one  consistent 
set  of  books  on  s  ires  by  all  agencies  and  is  reputed  to  have  largely 
solved  problems  of  over-counting. 
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Quantitative 


Yes.  Weight,  volume,  number  of  packages,  vehicle  characteristics, 
number  of  crew  members,  and  a  host  of  other  parameters  routinely  are 
listed  in  contraband  seizure  reports. 

Discriminating 

Technically  yes,  for  postulated  scenarios,^  where  simply  running 
the  simulation  or  exercise  a  little  longer  will  produce  sufficient 
sample  size,  but  often  not  for  real  operations.  The  lower  the  echelon, 
the  smaller  is  the  AOR  and  the  number  of  seizures.  Statistics  based  on 
a  small  number  of  events  have  large  standard  errors;  a  very  large  change 
in  the  seizure  statistics  is  required  to  have  any  confidence  that  an 
apparent  gain  is  a  real  effect,  not  just  the  product  of  noise  within  a 
wide  band  of  estimation  uncertainty.  The  best  chance  to  discern  signi¬ 
ficant  changes  with  justifiable  confidence  comes  at  the  highest  levels 
of  aggregation  (national,  across  all  agencies)  where  the  contribution  of 
lower  echelons  is  lost  from  sight. 

Realistic 

Only  in  part.  Seizures  provide  a  major  source  of  evidence,  and  the 
associated  arrests  put  the  accused  parties  in  custody,  but  the  real 
prize  for  law  enforcement  agencies  is  prosecution  and  conviction. 

Merely  capturing  drugs  is  a  rather  hollow  victory  because  the  smugglers 
can  easily  replace  their  losses. 

Objective 


No.  Optimists  say  rising  seizures  show  success  because  interdic¬ 
tion  forces  are  taking  more  drugs  out  of  circulation;  they  argue  that 
falling  seizures  show  success  because  the  smugglers  are  being  deterred. 
Skeptics  see  rising  seizures  as  signifying  failure  because  smugglers  are 
moving  more  drugs;  they  see  falling  seizures  as  further  evidence  of 
failure  because  the  smugglers  are  evading  law  enforcement  efforts .  The 
answer  depends  on  the  biases  one  brings  to  the  questions . 

Appropriate 

To  a  degree.  Seizures  made  in  an  AOR  are  real  events  and  represent 
an  achievement  for  the  forces  involved;  no  one  can  build  a  consistent 


1.  The  discussion  distinguishes  between  postulated  scenarios  and  real 
operations  where  appropriate.  The  absence  of  distinction  implies  no 
important  difference  between  the  two  contexts . 

2.  In  1988,  maritime  operations  in  the  entire  Atlantic  Command  AOR 
produced  only  9  to  12  seizures,  depending  on  specifics  of  the  counting 
rules.  Appendix  B  includes  a  detailed  summary  of  these  operational 
results . 


record  for  long  on  luck  alone,  but  the  opportunity  for  seizures  depends 
heavily  on  decisions  by  the  smugglers.  If  smugglers  make  a  major  shift 
in  their  regional  allocation  of  shipments,  the  amount  of  drugs  moving 
through  an  AOR  can  change  dramatically,  with  corresponding  changes  in 
seizure  statistics.  In  such  an  instance,  interdiction  forces  might 
deserve  neither  credit  nor  blame,  despite  large  charges  in  the  numbers. 

Sensitive 

In  part.  As  with  the  preceding  attribute,  the  number  and  size  of 
seizures  can  be  driven  by  smuggler  action  that  has  little  to  do  with 
effectiveness  of  the  interdiction  forces  involved. 

Promptly  Responsive 

No.  Despite  the  fact  seizures  are  reported  with  minimal  delay, 
they  occur  so  infrequently  that  results  need  to  accumulate  over  long 
periods  before  enough  data  exist  for  meaningful  comparisons  between 
periods . 

Simple  To  Understand 

Seemingly  so  on  the  surface,  but  not  upon  reflection.  Data  on 
seizure  events  and  weight  of  contraband  are  straightforward  and  easy  to 
grasp,  but  the  meaning  of  these  numbers  in  the  overall  scheme  of  the 
national  antidrug  strategy  is  much  less  clear.  In  particular,  seizure 
statistics  lie  in  a  kind  of  limbo  absent  comparable  figures  on  the  drug 
flow  from  which  the  seizures  are  drawn.  Seizures  per  se  are  statistics 
without  a  context;  neither  the  forces  on  the  front  line  nor  their 
leaders  in  Washington  have  a  sound  basis  for  interpretation. 

Validated 

No.  The  litany  of  problems  described  here,  especially  those 
regarding  objectivity  and  ease  of  understanding,  give  lie  to  any  asser¬ 
tion  that  seizures  have  been  validated  as  an  MOE. 

SUMMARY 

Raw  seizures  have  three  critical  failings,  any  one  of  which  is 
enough  to  discredit  its  candidacy  as  an  MOE  for  interdiction.  First, 
even  though  the  numbers  are  among  the  most  reliable  data  collected  on 
the  effects  of  interdiction,  they  have  little  meaning  taken  alone; 
changes  might  be  traced  to  many  different  causes,  few  of  which  have 
anything  to  do  with  effectiveness  of  interdiction  forces.  Second, 
evaluating  operations  on  the  basis  of  seizures  gives  a  powerful  incen¬ 
tive  to  focus  on  the  least  talented  segment  of  the  smuggling  threat  in 
order  to  squeeze  the  highest  score  from  limited  resources;  in  turn, 
sophisticated  smugglers  who  present  more  difficult  targets  stand  to 
benefit  from  systematically  receiving  less  than  their  share  of  atten¬ 
tion.  Finally,  experience  in  the  Caribbean  shows  that  seizures  occur  so 
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infrequently  in  the  maritime  arena  that  very  long  periods  must  elapse 
before  enough  data  accumulate  to  make  statistical  comparisons  capable 
discriminating  between  noise  and  real  effects;  this  latter  point  is 
explored  extensively  in  the  next  section  and  in  appendix  B. 


SECTION  5 


BOARDING  SUCCESS  RATE 


A  substantial  portion  of  interdiction  effort  is  directed  against 
smuggling  aboard  boats  and  ships  of  all  descriptions.  Coast  Guard  and 
Navy  vessels  carry  law  enforcement  detachments  (LEDETs)  composed  entire¬ 
ly  of  USCG  personnel  who  conduct  boardings,  seize  contraband,  and  arrest 
smugglers.  The  background  level  of  legitimate  traffic  far  outstrips  the 
ability  of  these  forces  to  visit  and  search;  they  must  be  judicious  in 
their  selection  of  targets,  exploiting  intelligence  cues  and  profiles  of 
smuggler  attributes  to  concentrate  effort  on  likely  suspects .  A  natu¬ 
rally  appealing  MOE  in  this  setting  is 


Boarding  success  rate  - 


Number  of  seizures 
Number  of  boardings 


It  relates  seizures  to  the  level  of  surface  maritime  interdiction  activ¬ 
ity  in  an  AOR.  It  is  a  measure  of  efficiency  because  results  are 
compared  with  interdiction  effort.  An  improvement  in  scores  from  one 
period  to  the  next  might  be  taken  as  an  indication  that  intelligence 
cues  and  boarding  criteria  are  becoming  better  targeted. 

CRITICAL  FLAWS 

In  a  naive,  idealized  sense,  the  MOE  would  signal  perfection  when 
every  boarding  produces  a  seizure.  An  obvious  way  to  attain  a  high 
boarding  success  rate  is  to  concentrate  on  targets  that  are  a  "sure 
thing."  One  plausible  consequence  is  excessive  attention  to  the  rela¬ 
tively  easy  targets  presented  by  smugglers  who  are  careless,  inexperi¬ 
enced,  or  untalented;  sophisticated  smugglers  with  their  more  elusive 
styles  could  end  up  operating  with  near  immunity.  The  news  releases 
might  look  good,  but  the  outcome  is  likely  to  be  counterproductive  in 
the  long  run. 

This  bias  toward  the  easy  targets  also  invites  a  circular  feedback 
process  tending  to  enshrine  a  narrow  range  of  intelligence  methods.  The 
quest  for  a  high  boarding  success  rate  encourages  concentration  on 
smugglers  tagged  through  top-quality  intelligence  cues;  each  new  seizure 
stands  to  reinforce  reliance  on  methods  that  produced  the  rewarding 
cues.  The  result  is  a  powerful  disincentive  against  any  activity  that 
might  dilute  the  boarding  success  rate:  pursuing  low-probability  cues, 
experimenting  with  new  methods ,  or  including  random  boardings  in  the 
CONOPS.  In  the  process,  opportunities  for  chance  discoveries  are 
narrowed,  if  not  eliminated,  providing  even  greater  security  to 
smuggling  organizations  whose  methods  successfully  evade  the  principal 
intelligence  indicators.  Such  an  outcome  provides  an  excellent  example 
of  how  a  plausible -sounding  idea  can  produce  unintended  results. 
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A  potential  remedy  to  this  problem  is  to  break  down  the  boardings 
according  to  the  quality  of  the  cue:  high,  low,  experimental,  random 
selection,  and  so  on.  Then  interdiction  forces  would  not  risk  a  poor 
showing  simply  as  a  result  of  varying  the  mix  of  reasons  for  their 
boarding  choices.  This  approach  has  two  drawbacks.  First  is  the 
temptation  to  game  the  process  by  tagging  unproductive  boardings  (after 
the  fact)  as  having  originated  with  low-probability  cues.  Second,  > 

breaking  the  data  stream  into  smaller  pieces  has  the  statistical  effect 
of  enlarging  the  uncertainty  bands  and  lengthening  the  time  necessary  to 
discriminate  real  changes  from  routine  variation.  The  example  in  the 
following  section  shows  that  this  statistical  problem  is  difficult  even 
without  subdividing  the  data. 

SAMPLE  APPLICATION  USING  HISTORIC  DATA 

One  good  thing  to  be  said  about  the  boarding  success  rate  is  that 
its  inputs  are  known  with  high  accuracy.  Despite  data  gaps  and 
inconsistencies,  the  counts  of  both  boardings  and  seizures  potentially 
can  be  tabulated  with  near  perfection.  Most  other  candidates  for 
interdiction  MOEs  depend  on  data  that  can  be  estimated  only  with 
substantial  uncertainty.  Even  with  this  powerful  advantage,  implausibly 
large  changes  in  this  MOE  would  be  required  before  it  could  signal  a 
real  change  in  effectiveness.  These  issues  are  illustrated  with  the 
following  highlights  from  a  detailed  analysis  presented  in  appendix  B. 

In  calendar  year  1988,  surface  drug  interdiction  operations  under 
Commander  Caribbean  Squadron  (COMCARIBRON) ^  executed  313  boardings  and 
made  9  to  12  seizures  (depending  on  the  counting  rules) --a  boarding 
success  rate  in  the  range  of  3  to  4  percent.  A  probability  model  can 
show  how  much  this  success  rate  must  increase  before  one  achieves  sta¬ 
tistical  confidence  that  there  has  been  a  real  effect.  The  required 
margin  of  difference  shrinks  with  increased  boardings  because  larger 
amounts  of  data  narrow  the  band  of  uncertainty.  As  compared  to  a 
4-percent  historic  success  rate,  table  5-1  shows  the  minimum  improvement 
in  the  boarding  success  rate  needed  for  the  difference  to  be  statisti¬ 
cally  significant  (one-sided  test,  5-percent  significance),  not  just  a 
chance  product  of  variability  in  smuggling  and  interdiction  activity. 

For  example,  30  boardings  (one  month  of  operations  at  the  1988 
level  of  activity)  would  have  to  produce  a  boarding  success  rate  over 
13  percent,  more  than  three  times  the  baseline  rate.  Even  a  year's 
worth  of  new  data  is  sufficient  only  if  the  change  in  the  MOE  exceeds 
58  percent.  Both  the  threat  and  interdiction  operations  are  dynamic;  - 

the  longer  the  time  frame,  the  greater  are  the  chances  that  an  observed 
change  is  due  to  some  uncontrolled  (and  perhaps  unrecognized)  effect, 
not  the  product  of  deliberate  operational  decisions.  These  large, 


1.  In  1988,  all  Atlantic  Fleet  surface  operations  dedicated  to  the 
antidrug  mission  were  under  the  tactical  control  of  COMCARIBRON,  a  Coast 
Guard  officer. 
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required  improvement  factors  are  the  direct  result  of  the  low  seizure 
frequency.  A  seizure  is  a  sufficiently  uncommon  event  that  even  fairly 
large  numbers  of  boardings  produce  few  successes;  as  a  result,  the 
statistical  properties  of  the  boarding  success  rate  naturally  result  in 
large  uncertainties. 


Table  5-1. 

Thresholds 

for  detecting  a 

change  in 

boarding  success  rate 

from  a  4-percent 

baseline 

Required 

Implied 

new 

Minimum 

Number  of 

time 

success  rate 

improvement 

boardings 

frame 

(percent) 

factor 

30 

1  month 

13.33 

3.33X 

100 

3  months 

8.00 

2.00X 

1  year 

6.33 

1.58X 

1,000 

3  years 

5.10 

1.28X 

BOARDING  SUCCESS  RATE  EVALUATED  ON  MOE  CRITERIA 
Mission  Oriented 

Yes,  in  part.  Clearly,  interdiction  forces  should  target  their 
boardings,  but,  if  operations  are  organized  to  maximize  the  boarding 
success  rate,  there  is  incentive  to  concentrate  on  the  highest  probabil¬ 
ity  targets  as  presented  by  the  least  capable  smugglers .  Effort 
expended  on  low-probability  targets  is  expected  just  to  lower  the  score. 
Indeed,  if  graded  on  just  this  MOE,  it  is  in  the  commander's  interest  to 
underutilize  the  boarding  capability  of  his  forces  unless  faced  with 
enough  good  targets  to  absorb  all  the  capacity.  A  similar  problem 
applies  when  using  the  measure  to  evaluate  the  quality  of  intelligence 
cues;  it  draws  attention  to  indicators  most  likely  to  produce  a  seizure, 
normally  those  associated  with  the  easiest  targets.  Sophisticated 
smugglers  who  present  ambiguous  cues  stand  to  be  systematically  ignored 
if  intelligence  support  is  graded  on  this  measure  alone.  Tagging  each 
boarding  with  the  quality  of  its  cue  offers  a  possible  solution,  but 
only  at  the  price. of  lessened  statistical  sensitivity  due  to  reduced 
sample  size. 

Measurable 

Yes.  Both  the  numerator  and  denominator  are  capable  of  precise 
measurement  by  the  forces  conducting  interdiction  operations.  The  key 
fault  to  avoid  is  multiple  counting  as  a  result  of  several  agencies 
reporting  a  single  boarding  or  seizure. 
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Quantitative 

Yes .  Both  numerator  and  denominator  are  numerical  and  represent 
sums  of  discrete  events. 

Discriminating 

Yes,  in  postulated  scenarios,  but  no  in  real  operations.  Even 
though  numbers  of  boardings  and  seizures  can  be  compiled  with  little  or 
no  reporting  error,  seizures  come  at  such  a  slow  pace  (at  least  those 
with  military  involvement)  that  sampling  error  produces  a  large  statis¬ 
tical  uncertainty  band,  but  this  effect  need  not  be  a  problem  in  postu¬ 
lated  scenarios  where  the  event  rate  can  be  controlled  as  an  independent 
variable . 

Realistic 

No.  As  with  several  other  criteria,  the  inherent  bias  toward  easy 
targets  draws  interdiction  forces  into  actions  that  look  good  but  in 
fact  are  counterproductive.  Even  more  damaging  is  the  incentive  for 
forces  to  reject  opportunities  that  do  not  offer  sufficient  prospect  of 
a  seizure  a  priori. 

Objective 

Technically  yes,  but,  practically,  no.  The  data  are  capable  of 
error-free  collection  and  the  MOE  computation  is  straightforward;  how¬ 
ever,  unless  offset  in  some  manner,  the  bias  toward  easy  targets  is  a 
challenge  to  assertions  of  objectivity. 

Appropriate 

Yes.  The  measure  is  fully  appropriate  to  commands  having  control 
over  both  the  choice  of  targets  for  boarding  and  the  performance  of 
search  parties.  If  these  functions  are  split  between  commands,  the 
measure  still  might  be  appropriate  to  their  common  parent  command,  if 
one  exists. 

Sensitive 


No.  At  first  glance,  this  measure  would  seem  to  deserve  a  yes  as 
it  directly  captures  the  results  of  boarding  activity,  but  if  interdic¬ 
tion  forces  concentrate  on  easy  targets  and  forgo  low-probability  tar¬ 
gets,  the  resulting  improvement  in  the  MOE  score  is  entirely  misleading. 
In  addition,  the  measure  puts  a  premium  on  two  elements  that  may  be 
outside  the  control  of  the  authorities  in  charge  of  the  interdiction 
operations:  intelligence  cues  and  the  boarding  party's  capability  to 
find  the  drugs  if  they  are  present  on  a  vessel. 
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Promptly  Responsive 

No.  Seizures  in  maritime  operations  are  sufficiently  rare  that 
long  periods  are  required  to  accumulate  enough  events  to  discriminate 
realistic  changes  in  performance.  Using  1988  operations  in  the 
Caribbean  as  a  reference  point,  the  boarding  success  rate  for  1989  would 
have  had  to  improve  by  at  least  58  percent  for  the  change  to  be  statis¬ 
tically  significant. 

Simple  To  Understand 

Maybe.  On  its  face,  the  measure  would  seem  to  warrant  a  yes 
answer,  but  the  subtlety  of  its  bias  toward  easy  targets  and  the  incen¬ 
tive  to  forgo  low-probability  opportunities  confuses  interpretation. 

Validated 

No.  The  measure's  performance  has  not  been  subjected  to  field 
testing. 

SUMMARY 

The  boarding  success  rate  provides  perverse  incentives  to  look  good 
at  the  expense  of  real  effectiveness  against  drug  smugglers.  Like  raw 
seizures,  the  measure  has  a  built-in  bias  toward  easy  targets,  but  it 
adds  another  dimension  through  discouraging  boardings  that  lack  high 
a  priori  expectation  of  a  seizure.  Commanders  have  an  incentive  to 
exploit  the  full  boarding  capabilities  of  their  forces  only  if  intelli¬ 
gence  produces  enough  high-probability  targets.  In  turn,  intelligence 
activities  have  an  incentive  to  concentrate  on  the  methods  that  produce 
the  highest-probability  cues,  giving  real  security  to  smugglers  who  can 
evade  the  principal  indicators.  Any  element  of  experimentation  or 
randomness  inherently  is  discouraged,  resulting  in  stylized  and  predic¬ 
table  operations . 

In  addition,  maritime  seizures  occur  with  such  low  frequency  that 
long  periods  are  needed  to  accumulate  enough  data  for  statistically 
meaningful  comparisons.  This  measure  illustrates  that  high-quality  data 
are  not  enough  to  produce  a  statistically  sensitive  measure;  small 
sample  size  stemming  from  a  low  event  rate  is  just  as  debilitating  as 
poor  data. 
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SECTION  6 


INTERDICTION  RATE 


Discussions  about  MOEs  for  interdiction  operations  often  conclude 
that  there  is  no  way  to  measure  the  amount  of  drugs  moving  through  an 
AOR  in  a  given  period.  Such  sentiments  usually  imply  the  speaker  has  in 
mind  the  interdiction  rate,  defined  as 


Interdiction  rate  - 


Seizures 

Shipments 


In  concept,  the  measure  can  be  applied  over  a  broad  range,  from  a  single 
patrol  of  an  individual  ship  to  multi-unit,  wide-area  operations  over 
extended  time  periods.  The  formulation  lends  itself  to  aggregation  over 
several  units  operating  in  parallel  as  in  a  barrier  operation;  handling 
sequential  effects  as  with  multiple  barriers  also  is  straightforward. 

The  interdiction  rate  is  a  measure  of  effectiveness  because  the  result 
is  in  terms  of  the  smuggler's  level  of  effort.  Counts  of  "seizures"  and 
"shipments"  typically  are  in  units  of  smuggling  vehicles  or  sorties; 
however,  some  simulations  and  analyses  in  postulated  scenarios  use 
weights  of  drug  shipments  and  seizures.  The  Coast  Guard  has  extensively 
investigated  the  interdiction  rate  and  its  application  [3,  14,  and  15]. 

CHARACTERISTICS  OF  THE  INTERDICTION  RATE  AS  AN  MOE 

In  typical  usage,  the  counts  of  seizures  and  shipments  are  tailored 
to  the  time  and  space  targeted  by  the  interdiction  units  of  interest; 
for  example,  in  the  case  of  a  temporary  barrier  at  sea, 


Interdiction  rate  - 


Smuggling  vessels  seized _ 

Smuggling  vessels  entering  barrier 


during  the  period  the  barrier  exists.  The  measure  is  figured  only  for 
the  time  and  space  where  interdiction  forces  put  smugglers  at  risk.  As 
a  result,  smuggler  action  to  wait  out  or  go  around  the  barrier  is  not 
reflected  in  the  MOE,  though  such  behavior  is  important  in  the  real 
world.  Neicher  is  there  any  perspective  on  how  meaningful  these  results 
might  be  from  the  smugglers'  frame  of  reference;  a  high  interdiction 
rate  in  one  area  may  not  be  significant  if  the  vast  majority  of  the 
contraband  is  moving  through  another  AOR.  The  next  section  examines  the 
smuggler's  rate  of  success  and  directly  addresses  these  issues. 

The  interdiction  rate  may  be  useful  as  a  criterion  in  postulated 
scenarios  for  operational  planning,  training  exercises,  and  computer 
simulations.  In  all  such  cases,  the  level  of  smuggling  activity  is  a 
stated  input;  therefore,  computed  interdiction  rates  are  known  perfectly 
(provided  the  scenarios  and  methods  used  to  compute  the  numbers  of 


6-1 


seizures  are  valid) .  These  relationships  are  fully  reversed  when 
figuring  the  interdiction  rate  for  real  operations;  seizures  are  known 
with  high  accuracy,  but  the  relevant  smuggling  activity  is  characterized 
by  estimates  with  large,  unknown  uncertainties.  The  validity  of  an 
interdiction  rate  computed  from  real  operations  rests  almost  entirely  on 
the  assumptions  and  methods  used  to  estimate  the  base  level  of  contra¬ 
band  shipments . 

Ideally,  the  estimate  of  relevant  smuggling  activity  in  the  denomi¬ 
nator  should  be  derived  independently  from  the  seizures  in  the  numeral 
tor,  but  making  a  direct  estimate  of  smuggling  activity  actually 
occurring  in  a  given  space  and  time  is  difficult.  Seizures  are  the 
highest-quality  evidence,  but  only  a  fraction  of  the  total;  this  frac¬ 
tion  is  the  interdiction  rate.  Additional  segments  of  smuggling  activ¬ 
ity  are  indicated  via  intelligence,  some  valid  and  some  not,  the  divi¬ 
sion  unknown.  Further,  these  data  sources  are  limited  to  the  smuggling 
activity  already  discovered;  the  part  that  is  unknown  could  be 
substantial,  but  it  is  inherently  not  measurable  on  any  current  basis. 

In  one  sense,  the  interdiction  rate  is  a  conservative  measure.  It 
credits  interdiction  forces  only  for  seizures .  Interdiction  activities 
have  other  effects  on  the  smuggler,  however: 

•  Increased  costs 

•  Delays  as  smuggling  missions  wait  for  more  attractive 
opportunities 

•  Missions  aborted  to  avoid  interdiction  forces 

e  Drugs  jettisoned  so  smugglers  avoid  being  caught  with  the 
evidence 

•  Heightened  risks  as  efforts  to  avoid  interdiction  increase 
exposure  to  effects  of  night,  weather,  incompetence,  and 
predatory  action  of  other  smugglers . 

These  effects  are  addressed  in  some  of  the  measures  introduced  in  later 
sections . 

INTERDICTION  RATE  EVALUATED  ON  MOE  CRITERIA 
Mission  Oriented 

Yes,  in  part,  but  with  important  qualifications.  If  the  duration 
or  coverage  of  the  interdiction  activity  is  too  narrow  in  scope,  the 
smuggler  may  have  the  option  to  wait  out  or  go  around  the  interdiction 
forces;  then,  even  an  impressive  interdiction  rate  is  liable  to  be 
distinctly  misleading,  because  the  apparent  local  success  pales  in  the 
larger  scheme  of  events.  On  the  other  hand,  if  the  concept  of  opera¬ 
tions  and  supporting  resource  commitment  produce  a  dense  and  continuing 
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presence  over  a  wide  area,  the  MOE  probably  comes  close  to  being  a 
capsule  statement  of  the  local  mission.  In  addition,  interdiction  rate 
shares  some  flaws  already  described  for  raw  seizures.  Both  measures 
offer  an  implicit  incentive  to  concentrate  on  the  easy  targets  and  score 
maximum  successes  for  a  given  level  of  effort,  while,  in  the  process, 
giving  the  sophisticated  smuggler  a  degree  of  immunity.  Further,  both 
measures  are  susceptible  to  manipulation  by  smugglers  who  deliberately 
sacrifice  some  shipments  as  decoys . 

Measurable 

Yes,  in  postulated  scenarios,  but  no  in  real  operations.  The  count 
of  seizures  in  the  numerator  is  knowable  with  high  precision,  but  the 
shipments  in  the  denominator  can  be  estimated  only  with  substantial 
uncertainty  in  the  real  world.  This  latter  problem  has  bedeviled  pro¬ 
ponents  of  the  interdiction  rate  for  years  as  some  unknown  amount  of 
smuggling  activity  simply  escapes  the  notice  of  surveillance  assets.  On 
the  other  hand,  the  MOE  is  eminently  employable  in  plan  development, 
simulations,  and  exercises  where  the  level  of  smuggling  activity  is  an 
independent  variable  under  control  of  the  investigator. 

Quantitative 

Definitely  yes  in  postulated  scenarios ,  but  in  real  operations ,  it 
is  hard  to  put  any  confidence  in  measurements  of  tht  smuggling  activity 
from  which  the  seizures  are  drawn.  The  tendency  for  opinion  or  intui¬ 
tion  to  shade  the  answers  is  hard  to  control  in  the  presence  of  large 
uncertainties . 

Discriminating 

Yes  in  postulated  scenarios  where  everything  is  known,  but  probably 
not  for  real  operations.  The  uncertainty  of  estimated  smuggling  activi¬ 
ty  in  real  operations  is  not  known  at  present,  though  it  would  be  sur¬ 
prising  if  the  margin  of  error  is  as  small  as  a  factor  of  two  for  a  full 
year  of  Caribbean  operations.  In  such  a  case,  the  interdiction  rate  for 
a  year  would  have  to  double  before  one  could  be  confident  that  the 
change  is  a  real  effect.  (The  uncertainty  factor  grows  even  larger  as 
one  attempts  to  evaluate  either  a  smaller  AOR  or  shorter  period  of 
interest. ) 

Realistic 

Yes,  with  qualifications.  With  the  background  level  of  smuggling 
as  a  base  of  reference,  the  interdiction  rate  is  more  realistic  than  raw 
seizures  as  an  MOE,  but  vulnerable  to  giving  a  distorted  answer  if 
intermittent  operations  leave  smugglers  with  exploitable  gaps.  It  does 
little  good  to  score  impressive  interdiction  rates  whenever  the 
operation  is  active  if  the  more  sophisticated  smugglers  are  able  to  move 
the  bulk  of  their  goods  during  periods  when  the  operation  is  suspended 
or  concentrated  elsewhere . 
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Objective 


Yes,  in  theory,  but  the  details  of  implementation  can  undermine 
objectivity  in  at  least  two  ways.  As  already  pointed  out,  a  judicious 
choice  of  the  domain  of  time  and  space  over  which  the  interdiction  rate 
is  computed  can  make  operations  look  undeservedly  good.  Perhaps  even 
more  difficult  are  the  uncertainties  in  estimating  the  underlying  level 
of  smuggling  activity.  Even  honest  disagreements  over  assumptions, 
data,  and  methods  can  result  in  estimation  differences  between  extremes 
of  dismal  and  outstanding. 

Appropriate 

Yes.  The  interdiction  rate  can  be  applied  at  any  scale,  from  small 
(a  single  unit  on  patrol  for  a  week)  to  grand  (the  national  aggregate  of 
all  forces  interdicting  drugs  in  a  year) ,  provided  a  sufficiently  reli¬ 
able  estimate  of  smuggling  activity  is  available. 

Sensitive 

Yes,  when  the  interdiction  forces  are  taken  collectively;  however, 
it  is  poorly  suited  to  assessing  single -agency  performance  in  a  multi¬ 
agency  operation  where  the  contribution  from  each  step  in  the  process 
depends  on  preparatory  and  follow-up  activities  of  the  other  partici¬ 
pants.  In  the  extreme,  perfect  performance  by  one  actor  can  be  offset 
fully  by  the  poor  performance  of  another. 

Promptly  Responsive 

No,  in  real  operations,  as  long  as  the  underlying  level  of 
smuggling  activity  remains  a  critical  uncertainty.  Though  it  is  diffi¬ 
cult  to  be  precise  in  the  absence  of  more  definitive  work,  this  problem 
of  uncertainty  almost  surely  requires  comparisons  over  relatively  long 
periods  to  detect  real  changes  in  performance. 

Simple  To  Understand 

Yes,  for  the  HOE  itself.  Interdiction  rate  is  an  easy  concept, 
having  pretty  much  the  same  meaning  to  operating  forces,  analysts,  and 
the  public;  however,  a  common  understanding  among  all  these  parties  will 
be  much  harder  to  achieve  regarding  statistical  uncertainty;  yet  this 
technical  issue  is  critical  in  determining  whether  an  apparent  change  in 
performance  has  any  validity. 

Validated 

No.  The  intellectual  and  practical  challenges  of  producing  valid 
estimates  for  the  level  of  smuggling  activity  in  a  specified  period  and 
AOR  have  yet  to  be  overcome. 
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SUMMARY 


The  biggest  problem  for  evaluating  real  operations  is  uncertainty 
in  measuring  the  underlying  magnitude  of  smuggling  through  the  area  and 
period  of  interest.  This  difficulty  might  seem  solved  in  postulated 
scenarios  where  both  the  level  and  character  of  smuggling  activity  are 
stipulated  as  inputs,  but  even  here,  statistical  uncertainty  is  replaced 
by  another  uncertainty  as  to  the  realism  of  modeling  interdiction 
forces,  smugglers,  and  their  interactions.  Three  other  problems  are 
more  subtle.  First,  even  though  interdiction  rate  is  defined  in  terms 
of  the  smugglers'  level  of  effort,  the  domain  of  its  implementation 
usually  is  defined  over  an  area  and  period  of  interest  tied  to  the 
interdiction  forces.  As  a  result,  the  measure  inherently  overlooks  any 
segment  of  the  smuggler  community  having  the  option  to  wait  out  or 
circumvent  the  interdiction  operation  of  interest.  Second,  the  measure 
retains  the  implicit  incentive  present  in  raw  seizures  to  concentrate  on 
the  easiest  targets.  Third,  the  measure  gives  little  insight  into  other 
positive  effects  of  interdiction  such  as  pushing  smugglers  into  risky 
behaviors . 
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SECTION  7 


SMUGGLER  SUCCESS  RATE 


The  measures  explored  in  the  three  previous  sections  concentrate  on 
finding  some  significance  in  seizures  of  smuggled  drugs,  as  defined  from 
the  point  of  view  of  the  interdiction  forces.  This  section  examines 
performance  from  the  opposite  fraae  of  reference,  that  of  the  drug 
smuggler.  The  result  is  a  time  measure  of  effectiveness  as  it  is  con¬ 
ceived  in  terms  of  interdiction's  impact  on  smugglers. 

INTERDICTION  FROM  THE  SMUGGLER'S  POINT  OF  VIEW 

The  immediate  objective  of  a  drug  smuggler  is  to  move  a  drug  ship¬ 
ment  from  the  source  country  into  the  U.S.,  all  the  while  avoiding 
intercept,  seizure,  and  arrest.  The  intervening  interdiction  forces  are 
important  to  him  primarily  because  they  could  deny  this  objective,  but 
the  threat  is  not  uniform  either  in  time  or  space.  The  gaps  and  weak¬ 
nesses  are  the  interesting  parts  for  him;  zones  of  strong  defense  are 
best  left  alone  unless  some  compelling  reason  forces  him  to  push  the 
drugs  through  anyway.  His  assessments  note  what  options  have  been 
denied,  but  prime  attention  goes  to  the  opportunities  remaining  open. 

Adopting  the  smuggler's  point  of  view  is  potentially  a  powerful 
device  for  evaluating  interdiction  operations;  the  aggregate  of  all 
interdiction  effort  is  seen  through  one  lens.  Differences  among 
regions,  agencies,  commands,  and  programs  are  of  little  interest  to  the 
smuggler  unless  they  affect  his  risks  and  costs  of  delivering  the  drugs. 
Irrelevant  detail  is  suppressed  and  attention  is  riveted  on  whether 
interdiction  operations  are  having  an  effect.  Planning  oversights, 
resource  shortages,  coordination  failures,  and  other  shortcomings  are 
scored  for  the  real  openings  they  offer  to  a  smuggler. 

PROPERTIES  OF  THE  SMUGGLER  SUCCESS  RATE 

The  rational  smuggler  should  avoid  unnecessary  risk.  He  can  take  a 
variety  of  actions  to  manage  his  exposure;  the  most  elementary  is  simply 
to  avoid  contact  with  interdiction  forces.  A  prime  tactic  is  to  blend 
with  the  background  and  exploit  the  fact  that  interdiction  forces  have 
the  capacity  to  inspect  only  a  small  fraction  of  all  the  traffic 
encountered.  Keeping  a  low  profile  depends  on  controlling  any  cues  that 
might  attract  unwanted  attention,  a  prescription  for  practicing  good 
operational  and  communications  security  (OPSEC  and  COMSEC) .  A  more 
active  approach  entails  close  study  of  interdiction  operations  to 
discover  and  exploit  their  weaknesses.  Further  tactical  measures 
include  tracking  the  location  and  status  of  interdiction  forces  for 
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real-time  evasion.^-  More  aggressive  smugglers  employ  OPDEC  measures  to 
distract  (and  even  engage  with  a  decoy)  interdiction  forces  while  the 
drug  shipment  goes  through.  Though  these  precautions  reduce  the  chances 
of  intercept  and  search,  a  risk- averse  smuggler  still  prepares  for  the 
worst.  A  highly  successful  tactic  is  to  separate  the  smuggler  and  the 
contraband- -hand  off  to  accomplices,  stash  in  a  safe  place,  or  jettison 
overboard  as  a  last  resort- -before  an  encounter  with  interdiction 
forces.  As  a  final  hedge,  smugglers  have  improved  their  concealment 
methods  to  the  point  that  they  can  afford  to  cooperate  with  boarding 
parties  [17],  believing  they  can  endure  a  search  with  little  risk,  short 
of  destructive  inspections  conducted  in  port. 

From  the  smuggler’s  point  of  view,  realized  risk  is  neatly  summar¬ 
ized  as  his  rate  (or  probability)  of  success.  This  measure  incorporates 
the  real  effects  of  all  factors  affecting  smuggling  operations.  In 
addition  to  seizures  and  arrests,  it  also  reflects  interdiction's 
indirect  effects,  such  as  inducing  the  smuggler  to  adopt  practices  that 
expose  him  more  forcefully  to  losses  from  weather,  his  own  ineptness, 
and  predation  by  other  drug  smugglers.  The  smuggler's  rate  of  success 
is  meaningful  within  any  geographic  slice  of  the  drug  traffic,  and 
results  are  suitable  for  aggregation  across  boundaries  to  produce 
measures  spanning  several  AORs.  From  the  interdiction  forces'  point  of 
view,  a  change  in  the  smuggler's  rate  of  success  is  a  direct  measure  of 
the  shifting  balance  of  resource  commitment  and  tactical  skill  between 
the  two  sides. 


Functional  Forms  of  the  Smuggler's  Rate  of  Success 

All  variants  of  the  MOE  are  of  this  basic  form: 


„  ,  Measure  of  drugs  landed  at  destination 

Smuggler  success  rate  -  — - 7 — “7 - 7--  . - — - 

Measure  of  attempted  drug  shipments 


1.  For  example,  in  Operation  Market  Time,  U.S.  and  Vietnamese  surface 
interdiction  forces  sought  to  search  the  largest  possible  number  of 
junks  in  the  surveillance  zone  along  the  coast.  To  make  maximum  use  of 
available  resources ,  they  adopted  a  rule  that  upon  completion  of  a 
search,  the  nearest  uninvestigated  vessel  would  be  the  next  one  boarded. 
North  Vietnamese  arms  smugglers  were  able  to  evade  simply  by  keeping  at 
least  one  vessel  between  themselves  and  the  patrol  boat  [16,  p.  21], 

2.  The  interdiction  rate,  examined  in  the  previous  section,  is  a  close 
cousin.  It  differs  from  the  smuggler's  rate  of  success  in  three 
respects:  the  perspective  is  that  of  the  interdiction  forces,  not  the 
smugglers;  it  is  defined  only  over  the  time  and  space  where  smugglers 
are  at  risk  to  interdiction;  and  it  credits  only  seizures,  ignoring 
other  effects. 
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The  denominator  reflects  the  magnitude  of  smuggling  activity  at  the 
origin,  and  the  numerator  captures  how  much  of  it  is  successful.  Though 
the  ultimate  destination  of  interest  is  the  U.S.  and  its  territories, 

,  some  evaluations  might  usefully  be  structured  around  intermediate  desti¬ 

nations  such  as  the  Bahamas  or  Mexico. 

The  term  "measure"  simply  refers  to  the  counting  rules  and  units  of 
•  expression  that  make  the  MOE  functional.  Basic  to  all  the  concepts 

offered  here  is  the  idea  that  a  smuggling  mission  begins  with  the 
initial  departure  of  a  load  of  drugs  and  ends  with  one  of  several  con¬ 
clusive  outcomes:  successful  delivery,  seizure  by  law  enforcement 
agents,  theft  by  pirates,  destruction  in  an  accident,  or  abandonment. 
Aborted  flights  and  sailings  are  merely  temporary  delays  along  the  way 
to  mission  completion. 

A  number  of  alternatives  for  measures  are  possible;  perhaps  several 
might  be  used  in  combination  to  illustrate  different  aspects  of  the 
problem : 

Missions 

Number  of  smuggling  missions  completed 
Number  of  smuggling  missions  attempted 

Though  the  aggregate  over  all  missions  (flights,  voyages,  shipments) 
might  be  computed,  more  meaningful  results  will  emerge  from  a  breakdown 
of  missions  by  drug  and  mode  of  transportation  (e.g.,  cocaine  via 
general  aviation  aircraft  or  marijuana  via  noncommercial  vessels) . 

Amount  of  drugs 

Amount  q f  dyugs  leaded  at.., destination 
Amount  of  drugs  in  attempted  shipments 

This  measure  necessarily  will  be  broken  down  by  drug.  An  aggregate  over 
drugs  of  different  types  would  be  meaningless  because  a  ton  of  marijuana 
and  a  ton  of  cocaine  are  vastly  different  in  physical  volume,  dollar 
value,  social  impact,  or  any  other  relevant  respect.  The  raw  results  of 
forming  ratios  based  on  weight  are  likely  to  be  somewhat  different  from 
those  obtained  using  missions  as  the  unit  of  account  simply  because 
shipments  come  in  a  range  of  sizes,  thereby  weighting  some  successful 
missions  more  heavily  than  others. 


1.  If  each  departure  following  an  abort  were  counted  as  a  new  mission, 
each  abort  would  be  valued  the  same  as  a  seizure,  theft,  accident,  or 
abandonment;  the  result  clearly  would  underestimate  the  smuggler's 
ability  to  deliver  drugs. 
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Dollar  Value 


Value  of  successful  drug  shipments 
Value  of  attempted  drug  shipments 

Ratios  based  on  tonnage  or  value  should  be  much  the  same  for  a  given 
drug,  unless  prices  exhibit  significant  regional  or  temporal  discrepan¬ 
cies.  Because  money  is  a  universal  denominator,  however,  this  measure 
might  support  a  variety  of  interesting  aggregates  across  drugs, 
smuggling  modes,  agencies,  commands,  and  geographic  areas.  The  hard 
part  may  be  managing  controversy  over  the  choice  of  prices,  particularly 
the  problem  of  assuring  that  different  drugs  are  valued  in  a  comparable 
manner.  The  usefulness  of  such  an  MOE  could  be  compromised  if  burdened 
with  explanations  and  qualifications  that  smother  the  message. 

SMUGGLER  SUCCESS  RATE  EVALUATED  ON  MOE  CRITERIA 

Mission  Oriented 

Yes.  In  the  abstract,  this  measure  is  the  most  mi.®  .ion  oriented  of 
the  six  reviewed  in  this  paper.  It  addresses  precisely  the  prime  opera¬ 
tional  activity  of  the  interdiction  force,  attrition  of  the  smuggler. 

It  goes  further  than  the  interdiction  rate  by  taking  account  of  all 
kinds  of  losses  the  smuggler  may  suffer,  not  just  seizures  by  law 
enforcement  agents.  The  measure  is  so  inclusive,  however,  that  it  is 
suitable  only  for  evaluating  the  collective  performance  of  all  agencies 
acting  in  the  AOR  and  period  of  interest;  for  a  single  agency,  the 
measure  is  sensible  only  when  that  agency  has  an  exclusive  franchise 
over  the  AOR.  Earlier  measures  have  been  seen  to  encourage 
concentration  on  the  easy  targets;  applications  of  the  smuggler  success 
rate  can  address  this  problem  quite  naturally  by  dividing  the  smuggler 
population  into  strata  or  layers.  For  example,  different  rates  might  be 
computed  for  novices  and  professionals,  explicitly  recognizing  the 
sophisticated  smuggler  as  a  more  difficult  target.  Finally, 
interdiction  successes  against  decoy  shipments  are  more  likely  to  be 
recognized  as  marginal  events  when  seen  from  the  smuggler’s  point  of 
view  rather  than  from  that  of  the  interdiction  force. 

Measurable 

Yes ,  in  postulated  scenarios ,  but  no  in  real  operations .  There  is 
no  reasonable  hope  of  collecting  sufficient  data  to  make  precise  esti¬ 
mates  of  actual  smuggler  shipments  and  deliveries  in  a  chosen  AOR.  The 
MOE  does  fit  this  criterion,  however,  when  used  in  postulated  scenarios 
such  as:  CONOPS  development,  simulations,  exercises,  or  evaluation  of 
operations  against  assumed  smuggler  profiles.  In  all  of  these 
instances,  smuggler  shipments  are  an  input  variable  and  smuggler  deliv¬ 
eries  are  the  product  of  analysis  or  measurement.  Such  applications 
require  extraordinary  care  and  skepticism  in  developing  the  methodology 
to  avoid  bogus  results. 
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Quantitative 

Yes ,  but  with  qualifications .  In  real  operations ,  the  data  are 
quantitative  only  in  theory,  because  they  are  not  collectable  to  any 
useful  extent.  When  applied  in  postulated  scenarios,  the  necessary  data 
are  quantitative,  though  with  the  qualification  that  they  are  no  better 
than  the  quality  of  the  models  that  produce  the  numbers. 

Discriminating 

Potentially  yes.  For  postulated  scenarios,  any  final  judgement 
must  rest  on  a  critical  evaluation  of  model  implementation,  data  quali¬ 
ty,  and  the  specific  manner  in  which  the  models  are  employed  to  investi¬ 
gate  performance  and  potential  weaknesses  in  operations.  In  real  opera¬ 
tions,  the  uncertainty  of  estimated  smuggling  activity  is  not  known  at 
present,  though  it  would  be  surprising  if  the  margin  of  error  is  as 
small  as  a  factor  of  two  for  a  full  year  of  Caribbean  operations;  then 
the  interdiction  rate  for  a  year  would  have  to  double  before  one  could 
be  confident  that  the  change  is  a  real  effect,  not  mere  noise.  (The 
uncertainty  factor  grows  even  larger  as  one  attempts  to  evaluate  either 
a  smaller  AOR  or  shorter  period  of  interest.) 

Realistic 

Yes,  in  postulated  scenarios,  if  models  are  adequately  representa¬ 
tive.  In  real  operations,  the  MOE  is  realistic  in  principle,  but  that 
benefit  is  of  no  practical  value  because  the  required  data  are  not 
available  under  any  foreseeable  conditions. 

Objective 

Yes,  for  postulated  scenarios,  but  in  a  mechanical  way,  because 
results  can  only  be  the  product  of  a  model.  Not  likely  for  real  opera¬ 
tions  given  the  broad  uncertainty  in  the  baseline  quantity  of  smuggler 
shipments . 

Appropriate 

Yes,  if  applied  at  a  level  that  includes  all  interdiction  activity 
in  the  period  and  AOR  of  interest,  a  condition  that  probably  means  the 
activities  of  several  agencies  are  inseparably  intertwined. 

Sensitive 

Potentially  yes  in  postulated  scenarios.  The  actual  degree  of 
sensitivity  will  depend  on  characteristics  of  the  models  and  their 
implementation;  any  validation  process  should  make  a  special  point  of 
assessing  this  property.  As  for  real  operations,  uncertainty  in  the 
estimate  of  smuggler  shipments  virtually  rules  out  any  chance  of  the  MOE 
being  validly  sensitive  to  changes  in  CONOPS. 
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Promptly  Responsive 

No.  Whether  the  passage  of  time  is  real  or  simulated,  Interdiction 
events  occur  too  slowly  for  rapid  feedback. 

Simple  To  Understand 

Yes.  The  MOE  itself  is  simple  and  easy  to  grasp.  The  hard  part 
comes  in  establishing  confidence  in  estimates  of  real  smuggling  activity 
or  in  demonstrating  that  models  are  adequate. 

Validated 

Not  yet  for  postulated  scenarios;  the  posited,  linked  models  must 
be  constructed  and  thoroughly  tested  before  putting  faith  in  the  idea 
that  their  output  will  be  useful.  Validation  of  the  MOE  for  real  opera¬ 
tions  appears  out  of  the  question. 

SUMMARY 

Taking  the  smuggler's  point  of  view  is  potentially  a  powerful 
device;  it  rolls  together  all  interdiction  activity  in  the  area  and 
period  of  interest  and  draws  attention  to  the  aspects  smugglers  can 
exploit.  Measuring  performance  in  these  terms  would  strongly  reinforce 
understanding  of  the  mission.  Offsetting  these  virtues  is  the  fact  that 
the  MOE  has  no  value  for  real  operations  because  the  data  are  not  col¬ 
lectable  in  any  practical  sense.  Nevertheless,  the  concept  is  suitable 
for  postulated  scenarios  such  as  games,  simulations,  and  exercises. 
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SECTION  8 


IMPORT-EXPORT  PRICE  DIFFERENCE 

* 


The  rewards  of  drug  smuggling  are  profits.  Remove  the  financial 
gain,  and  the  incentive  to  deal  in  drugs  dissipates,  even  if  risks  fade 
*■  at  the  same  time.  Whether  smugglers  are  considered  individually  or  as  a 

group,  their  actual  costs  and  profits  are  not  susceptible  to  direct 
measurement.  However,  Peter  Reuter  and  colleagues  at  the  Rand  Corpora¬ 
tion  [18]  proposed  the  difference  between  import  and  export  prices  as  a 
closely  related  measure  and  used  it  in  their  high-level  policy  analysis 
of  whether  interdiction  could  be  effective.  This  section  explores 
whether  such  a  measure  might  have  some  utility  for  evaluating  operation¬ 
al  performance. 

THEORETICAL  FOUNDATION 

The  import  price  is  realized  during  the  first  transaction  after  the 
drugs  are  landed  in  the  U.S.;  the  export  price  is  the  smuggler’s 
purchase  cost  in  the  source  country.  The  difference  between  the  two 
reflects  transportation  costs,  risk  premiums,  and  smuggler  profits. 

(The  Rand  study  assesses  the  effects  of  interdiction  on  export  prices  as 
sufficiently  small  to  be  neglected  safely.) 

All  costs  are  passed  through  to  the  customer,  but  only  in  the  long 
run  and  on  average.  A  smuggling  ring  that  suffers  a  seizure  cannot 
raise  prices  to  recover  added  costs  if  its  customers  can  turn  to  alter¬ 
native  sources  that  are  not  experiencing  the  same  cost  burden.  Like¬ 
wise,  the  class  of  smugglers  at  risk  to  interdiction  cannot  raise  prices 
to  recover  fully  their  added  costs,  because  any  price  increase  will 
induce  greater  supply  from  other  transport  modes  not  exposed  to  the  same 
risk  (e.g.,  mail,  commercial  cargo,  body  carry).  At  equilibrium,  the 
smugglers  suffering  the  effects  of  interdiction  will  see  both  a  price 
that  covers  only  some  fraction  of  their  increased  costs  and  a  smaller 
market  share;  therefore,  the  difference  between  import  and  export  prices 
will  grow,  but  by  something  less  than  the  full  increase  in  costs  to  the 
affected  smugglers.  Also,  as  their  market  share  declines,  a  falling 
fraction  of  the  costs  imposed  by  interdiction  will  show  up  in  the  price 
difference,  in  the  extreme  reaching  zero  if  drug  smugglers  shift  their 
entire  flow  to  transport  methods  not  subject  to  interdiction.  These 
perturbations  take  time  to  work  their  way  through  the  market,  the 
effects  appearing  with  some  lag. 

SUITABILITY  FOR  EVALUATING  OPERATIONAL  PERFORMANCE 

Given  these  characteristics ,  is  there  any  utility  in  using  the 
*  import-export  price  difference  as  a  measure  of  operational  performance? 

On  the  positive  side,  price  is  a  powerful  mechanism  for  summarizing  all 
factors  with  an  economic  effect:  smuggler  costs,  alternative  smuggling 
methods,  risk  of  arrest,  consumer  response,  and  so  on.  The  Rand  study 
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concludes,  however,  that  smuggling  costs  and  profits  compose  only  about 
10  percent  of  the  Import  price  for  cocaine.  If  interdiction  is  intensi¬ 
fied  to  the  point  that  it  doubles  the  costs  of  smuggling,  the  import 
price  could  increase  by  10  percent,  at  most,  and  probably  somewhat  less 
since  other  transport  methods  have  significant  market  share.  Is  it 
possible  to  detect  with  confidence  a  price  change  of  a  few  percent  in  a 
useful  time  frame? 

The  Drug  Enforcement  Administration  collects  data  from  undercover 
negotiations  and  completed  buys,  compiling  the  results  quarterly.  The 
Rand  study  examined  these  data  and  found  them  of  sufficiently  doubtful 
quality  that  they  warranted  treatment  with  only  the  most  robust  of 
methods.  That  study  presents  data  [18,  appendix  B]  showing  that  the 
interquartile  range  (the  band  containing  50  percent  of  the  data, 
centered  around  the  median)  is  about  20  to  25  percent  of  the  median,  on 
average ;  that  is ,  if  cocaine ' s  median  import  price  during  a  quarter  was 
$25,000  per  kilogram,  half  of  the  data  should  fall  in  a  range  of  about 
$22,000  to  $28,000.  To  continue  the  example,  assume  the  median  export 
price  is  $4,000,  with  an  interquartile  range  of  $3,500  to  $4,500.  The 
difference  between  import  and  export  prices  then  has  a  median  of  $21,000 
and  an  interquartile  range  of  $18,500  to  $23,500;  these  numbers  are 
shown  as  the  baseline  case  in  table  8-1. 


Table  8-1.  Effects  of  interdiction  on  cocaine  prices 


Export 

price 

_ Baseline  case _ 

Smuggling 

Import 

price 

Import- 

export 

difference 

Import 

price 

Import- 

export 

difference 

Upper 

4,500 

28,000 

23,500 

30,800 

26,300 

quartile 

Median 

4,000 

25,000 

21,000 

27,500 

23,500 

Lower 

3,500 

22,000 

18,500 

24,200 

20,700 

quartile 

Now  consider  the  significance  if  some  change  in  the  CONOFS  doubles 
the  cost  of  smuggling  and  the  next  quarter  shows  a  full  10-percent 
increase  in  the  import  price  (median  of  $27,500  and  interquartile  range 
of  $24,200  to  $30,800)  while  the  export  price  figures  remain  steady. 
Standard  nonparametric  statistical  tests  are  designed  to  operate  on  the 
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raw  data,^  not  on  summaries  as  in  table  8-1;  however,  some  insight  can 
be  gained  from  thinking  of  the  interquartile  ranges  as  50-percent  confi¬ 
dence  intervals,  illustrated  in  figure  8-1. 


$23,500 

$21,000 

$18,500 


$26,300 

$23,500 

$20,700 


Baseline  Doubled  costs 


Figure  8-1.  Hypothetical  comparison  of  import-export  difference  when 
interdiction  doubles  smuggling  costs 


The  import  price  increase  moved  the  median  for  the  second  quarter 
just  to  the  edge  of  the  first  quarter’s  interquartile  range.  As  a 
result,  there  is  about  a  50-50  chance  that  the  second  quarter’s  import - 
export  price  difference  is  not  a  significant  divergence  from  the  first, 
the  apparent  change  being  merely  a  result  of  sampling  variation  between 
the  two  periods.  Furthermore,  statistical  decisions  commonly  rely  on 
confidence  intervals  of  90  or  95  percent,  not  50  percent;  the  change  in 
the  price  difference  would  have  to  be  two  or  three  times  larger  than  in 
the  example  to  satisfy  this  more  typical  standard.  In  sum,  it  seems 
unlikely  that  a  change  in  the  effectiveness  of  interdiction  so  dramatic 
as  to  double  the  costs  of  smuggling  would  result  in  a  significant  change 
in  the  import-export  price  difference  from  one  quarter  to  the  next. 

Taking  a  longer  view,  data  from  several  quarters  preceding  and 
following  a  change  in  CONOPS  might  supply  acceptable  evidence  of  a  break 
in  the  price  trend,  even  though  changes  between  successive  periods  are 
modest.  Even  if  a  correlation  can  be  established,  there  still  remains 
the  question  of  whether  the  apparent  relationship  is  one  of  cause  and 
effect,  or  merely  coincidence.  As  already  noted,  import  modes  not 
subject  to  interdiction  hold  a  sizable  fraction  of  the  market;  actions 
by  either  smugglers  or  LEAs  in  these  other  segments  have  the  power  to 


1.  The  Wilcoxin  rank  sum  test  is  commonly  used  for  deciding  whether  the 
medians  of  two  random  samples  are  significantly  different.  Its 
theoretical  foundation  and  practical  application  are  described  in 
standard  texts  on  nonparametric  statistics  [e.g.,  19]. 
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obscure  Che  price  effect  stemming  from  interdiction.  As  a  result,  a 
practical  system  would  need  to  be  broadly  conceived,  assessing  the  price 
impact  of  all  market  sectors ,  including  an  important  domestic  component 
for  products  such  as  marijuana. 

IMPORT-EXPORT  PRICE  DIFFERENCE  EVALUATED  ON  MOE  CRITERIA 
Mission  Oriented 

Probably  not.  Interdiction  forces  likely  would  have  difficulty 
seeing  how  their  operations  affect  the  import-export  price  difference. 
Even  estimating  their  impact  on  the  smugglers'  costs  is  likely  to  entail 
large  uncertainties.  As  a  result,  the  MOE  would  not  supply  much  guid¬ 
ance  for  commanders  trying  to  choose  among  alternative  strategies  or 
deployments . 

Measurable 

Yes,  for  postulated  scenarios,  but  the  issue  is  whether  the 
modeling  is  good  enough  to  make  the  results  of  any  value.  Potentially 
yes  for  real  operations.  If  LEAs  considered  it  worth  the  effort,  data 
quality  might  be  improved  substantially.  Not  only  would  it  be  necessary 
to  reduce  the  incidence  of  recording  and  handling  errors,  but  also  a 
substantial  effort  would  be  needed  to  correctly  identify  the  stage  of 
drug  distribution  represented  in  each  reported  transaction. 

Quantitative 

Potentially,  yes.  The  MOE  gives  a  measured  numerical  result,  but 
the  it. sue  is  whether  the  numbers  are  to  be  believed.  The  Rand 
researchers  found  existing  data  did  not  deserve  much  better  than  quali¬ 
tative  treatment.  Though  the  problem  is  different,  building  worthwhile 
models  for  use  in  postulated  scenarios  is  likely  to  be  as  great  a  chal¬ 
lenge  as  collecting  adequate  data  from  the  real  world. 

Discriminating 

No,  at  least  with  the  data  quality  presently  available.  The 
example  presented  above  showed  that  a  doubling  of  smuggler  costs  would 
have  an  effect  on  the  MOE  that  is  much  too  small  to  discern  within  the 
quality  of  available  data. 

Realistic 

No.  The  relationship  between  interdiction  operations  and  the  MOE 
almost  surely  will  be  obscure  to  the  deployed  forces. 
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Objective 


Probably  not.  The  stage  of  the  drug  distribution  process  repre¬ 
sented  in  each  transaction  is  often  difficult  to  determine.  Alternative 
criteria  for  selecting  the  appropriate  data  points  will  produce 
different  results. 

Appropriate 

No.  The  import-export  price  difference  is  the  joint  result  of  the 
balance  among  all  antidrug  activities  and  all  sources  of  illegal  drugs, 
much  of  which  is  beyond  the  influence  of  interdiction. 

Sensitive 

No.  Interdiction  acts  principally  to  raise  smuggling  costs,  an 
element  the  Rand  researchers  estimate  as  amounting  to  only  10  percent  of 
the  import-export  price  difference.  Even  an  interdiction  campaign  so 
severe  as  to  double  smuggling  costs  could  increase  the  MOE  by  only 
10  percent. 

Promptly  Responsive 

No.  The  Rand  researchers  found  that  quarterly  data  were  of  barely 
useful  quality;  even  with  good  data,  the  small  role  of  smuggling  costs 
makes  it  likely  that  the  MOE  could  detect  only  implausibly  large  changes 
in  a  few  quarters. 

Simple  To  Understand 

Not  likely.  Personnel  not  schooled  in  economics  probably  would 
have  difficulty  grasping  the  connection  between  interdiction  operations 
and  the  import- export  price  difference.  In  addition,  outside  the  realm 
of  interdiction,  numerous  activities  of  both  smugglers  and  LEAs  can 
affect  the  difference. 

Validated 

No.  The  MOE  is  no  more  than  a  hypothetical  concept. 

SUMMARY 

In  summary,  three  key  difficulties  undermine  the  import-export 
price  difference  as  a  prospective  MOE  for  operational  evaluation. 

First,  smuggling  costs  are  such  a  small  element  of  import  prices  that 
even  large  increases  have  little  apparent  effect.  Second,  current  data 
on  relevant  prices  seem  too  low  in  quality  to  discriminate  plausible 
effects  when  comparing  successive  quarters.  Third,  the  import-export 
price  difference  for  a  given  drug  is  the  joint  result  of  all  sources 
acting  together,  only  some  of  them  subject  to  interdiction.  An  assess¬ 
ment  system  based  on  this  measure  would  have  to  be  comprehensive  so  it 
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could  allocate  all  effects  to  sources;  its  sponsorship  and  administra¬ 
tion  clearly  would  be  beyond  the  reach  of  the  interdiction  community 
acting  alone. 
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SECTION  9 


MARKET  SHARE 


In  the  extreme,  a  drug  smuggler  might  be  deterred  when  hazards 
become  intolerable  or  profits  evaporate,  but  long  before  reaching  this 
point,  a  rational  smuggler  experiencing  increased  pressure  should  move 
into  alternatives  offering  a  better  mix  of  risks  and  rewards .  Historic 
adaptations  to  interdiction  success  are  instructive  in  this  regard. 
Smuggling  from  Latin  America  has  shifted  away  from  Caribbean  routes  and 
into  Mexico.  Transport  methods  have  changed  from  direct  delivery  to 
multistage  movements.  More  drugs  are  moving  through  ports  of  entry 
concealed  in  commercial  cargo.  Farther  afield,  smugglers  have 
encouraged  new  markets  in  Europe  and  the  Far  East.  They  have  even 
opened  other  lines  of  business  that  complement  the  drug  trade  or  capi¬ 
talize  on  its  skills  and  resources.  The  very  fact  of  such  adaptations 
is  sometimes  cited  as  evidence  that  interdiction  forces  are  effective 
[e.g.,  20]. 

This  line  of  thought  suggests  that  changes  in  the  allocation  of 
smuggling  activity  among  alternatives  might  give  some  insight  into 
whether  interdiction  is  doing  any  good.  For  example,  today's  marijuana 
smuggling  in  the  Caribbean  is  only  a  shadow  of  its  former  self;  though 
substantial  activity  remains,  the  traffic  has  shrunken  markedly  and  its 
particulars  have  changed  over  the  past  decade.  Movements  over  the 
Mexican  border  combined  with  domestic  production  have  become  the  domi¬ 
nant  factors  in  a  notably  enlarged  supply  [21]. 

One  possibility  for  quantifying  such  changes  is  to  track  the  market 
share  held  by  smuggling  methods  subject  to  interdiction.  This  measure 
would  seem  to  capture  the  direct  effects  of  removing  drugs  from  the 
pipeline,  price  impacts  from  moving  to  alternative  smuggling  methods, 
and  deliberate  actions  by  smugglers  to  shift  the  mode  or  location  of 
their  activities. 

IMPLEMENTATION  ISSUES 

Tracking  market  shares  entails  estimating  the  quantities  moved  via 
the  methods  and  regions  subject  to  interdiction,  complemented  by  compa¬ 
tible  estimates  of  all  other  flows  of  that  same  drug  into  the  country. 
However  plausible  this  notion  may  sound,  experience  with  national  drug 


1.  When  seen  in  this  larger  context,  the  value  of  interdiction's 
apparent  success  is  deflated;  smuggling  is  down  where  interdiction  has 
been  at  work,  but  the  overall  problem  is  worse  than  ever.  Taken  alone, 
the  deterrent  effect  of  interdiction  may  be  no  more  than  local,  simply 
displacing  traffic  to  safer  avenues.  Antidrug  efforts  will  encounter 
this  frustration  as  long  as  a  market  exists  and  smugglers  can  find  (or 
create)  gaps  in  the  shield  of  interdiction  and  law  enforcement. 


estimates  from  the  National  Narcotics  Intelligence  Consumers  Committee 
(NNICC)  [21]  suggests  that  the  kind  of  statistics  presently  available 
will  not  provide  useful  results.  Four  factors  support  this  conclusion. 

First,  the  regional  distribution  of  market  share  is  not  compiled 
currently  in  any  systematic  way.  NNICC  estimates  of  illegal  drug 
supplies  are  the  results  of  two  macro-scale  approaches.  One  method 
begins  with  global  surveys  of  raw  materials  (acres  of  coca,  marijuana, 
or  opium  poppies)  and  computes  outputs  at  each  stage  of  the  production 
process  to  estimate  aggregates  of  finished  drug  supplies  available  in 
the  U.S.  The  other  method  starts  with  surveys  of  the  user  population 
and  then  develops  estimates  for  consumption  and  the  magnitude  of  the 
associated  market. 

Second,  any  bureaucratically  realistic  approach  would  either  have 
to  be  integrated  with  the  current  NNICC  process  or  replace  it 
outright- -the  agencies  involved  could  not  afford  the  resources  for  an 
independent  alternative.  References  [22]  and  [23]  detail  shortcomings 
in  the  NNICC  methods,  many  of  them  related  to  inadequate  treatment  of 
uncertainty.  Figures  usually  are  reported  as  point  estimates;  occasion¬ 
ally  a  range  is  cited  but  without  explanation  as  to  what  it  means. 

Absent  a  quantitative  understanding  of  uncertainty  in  the  estimates, 
there  is  little  basis  for  judging  whether  a  change  in  an  estimate  from 
one  year  to  the  next  is  a  signal  or  just  routine  noise.  Reference  [22] 
gives  a  clear  idea  of  the  difficulties  in  making  the  estimates  and  the 
even  greater  challenge  of  characterizing  uncertainty.  These  problems 
for  national  aggregates  will  be  multiplied  not  only  in  number,  but  also 
in  severity,  if  the  process  is  refined  to  produce  component  estimates 
broken  down  by  smuggling  method  and  region.  The  prospects  for  producing 
useful  market -share  data  are  dismal  without  some  dramatic  improvement  in 
data  and  methods. 

Third,  even  if  annual  estimates  are  released  on  schedule  (one  or 
two  quarters  into  the  following  year) ,  they  still  are  not  timely  feed¬ 
back  to  the  chain  of  command  as  to  the  success  or  failure  of  operations. 
Shortening  the  time  slice  to  quarterly  or  monthly,  however,  would  shrink 
sample  size  and  exacerbate  the  uncertainty  problem. 

Fourth,  interpreting  changes  in  market  share  is  likely  to  present 
formidable  problems  (assuming  estimates  can  be  made  with  enough  accuracy 
to  justify  the  effort).  Movements  in  market-share  statistics  are  rela¬ 
tive;  a  rise  or  fall  in  the  target  sector's  market  share  depends  as  much 
on  what  is  happening  in  other  sectors  as  it  does  on  interdiction's 
effectiveness . 

All  in  all,  market  share  is  liable  to  be  a  blunt  tool,  rarely 
giving  confident  signals;  if  it  does  signal  a  major  shift,  the  event  is 
likely  to  be  apparent  without  going  to  all  the  trouble  of  computing 
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regional  market  shares.  This  combination  of  insensitivity  and  enormous 
estimation  problems  leaves  the  measure  as  having  little  more  than  theo¬ 
retical  interest  for  evaluating  operations. 

MARKET  SHARE  EVALUATED  ON  MOE  CRITERIA 

Mission  Oriented 

Yes,  for  postulated  scenarios,  but  only  if  corrected  for  all  exter¬ 
nal  effects.  No,  for  real  operations.  Reducing  the  market  share  for  an 
AOR  is  a  step  forward,  provided  the  gain  is  not  compensated  by  increased 
smuggling  activity  in  other  areas ,  a  phenomenon  known  among  LEAs  as 
displacement.  Progress  entails  reducing  the  size  of  the  entire  market, 
not  just  moving  the  pieces  around. 

Measurable 

Possibly,  for  postulated  scenarios,  if  the  process  can  be  modeled 
credibly.  Not  likely  for  real  operations.  Current  methods  of 
estimating  the  annual  size  of  the  national  market  are  fraught  with 
uncertainty;  results  of  doing  the  same  for  regional  shares  on  shorter 
time  scales  would  have  the  quality  of  a  fairy  tale  without  a  substantial 
improvement  in  data  and  methods. 

Quantitative 

Yes,  for  postulated  scenarios,  but  trivially  so  unless  models 
conform  well  to  reality.  Probably  not  for  real  operations.  Though 
market  share  is  numeric  in  concept,  believable  figures  suitable  for 
evaluating  real  operations  seem  beyond  the  reach  of  estimation  methods 
currently  employed. 

Discriminating 

Yes,  for  postulated  scenarios,  but  in  an  artificial  way;  models  can 
produce  sample  sizes  unobtainable  in  the  real  world.  No,  for  real 
operations.  A  key  problem  with  even  the  best  statistics  about  market 
size  is  uncertainty;  though  error  bands  rarely  are  reported,  qualitative 
indications  suggest  they  should  be  large. 

Realistic 

No.  Market  share  has  little  connection  to  the  operational  situa¬ 
tion  as  seen  from  the  perspective  of  interdiction  forces . 

Objective 

Yes,  for  postulated  scenarios,  provided  the  models  are  sufficiently 
realistic.  Indeterminate  for  real  operations,  in  the  absence  of  any 
realistic  prospect  of  computing  market  shares. 
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Appropriate 

No.  Market  share  in  one  sector  depends  not  only  on  the  outcome  of 
the  contest  there,  but  also  on  results  in  other  sectors  as  well. 

Sensitive 


No.  A  deliberate  smuggler  decision  to  shift  weight  of  effort  to 
methods  not  subject  to  interdiction  (e.g.,  the  commercial  container 
trade)  would  have  a  large  impact  on  market  shares  and  yet  have  nothing 
to  do  with  the  effectiveness  of  interdiction. 

Promptly  Responsive 

No.  The  only  official  estimate  of  the  overall  size  of  the  drug 
market  appears  as  annual  data,  and  that  report  is  released  one  or  two 
quarters  after  the  fact.  It  would  take  a  large  additional  effort  to  do 
even  this  well  with  regional  market  share  estimates. 

Simple  To  Understand 

No.  The  link  between  interdiction  operations  and  a  subsequent 
estimate  of  market  share  is  likely  to  be  obscure  and  complicated  because 
the  result  is  subject  to  many  external  influences  as  well  as  a  broad 
range  of  uncertainty. 

Validated 


No.  Market  share  as  a  measure  of  operational  performance  is  no 
more  than  a  hypothetical  concept. 

SUMMARY 

Reliable  data  showing  the  allocation  of  market  share  over  smuggling 
regions  probably  would  be  useful  for  gaining  a  strategic  perspective, 
but  market  share  seems  to  have  little  potential  as  an  MOE  for  evaluating 
interdiction  operations  on  a  regional  basis.  The  chief  problem  is  that 
market  shares  can  change  due  to  a  host  of  factors,  interdiction  effec¬ 
tiveness  in  s">me  target  region  being  only  one  of  many.  A  more  practical 
problem  is  the  poor  prospect  for  estimating  market  shares  with  suffi¬ 
cient  precision  to  tell  when  a  real  change  occurs.  Current  estimates 
offer  breakdowns  no  lower  than  the  national  market  and  no  more  often 
than  annual;  yet,  even  this  degree  of  aggregation  is  difficult  to 
compile  and  leaves  large  uncertainties . 
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SECTION  10 


EXAMPLE  OF  A  SUITE  OF  MOEs 


Of  the  six  MOEs  reviewed  in  preceding  sections,  none  proved  to  be 
winners  on  all  counts;  however,  combinations  might  be  attractive  if  one 
MOE  can  neutralize  liabilities  of  another.  Combinations  might  also  be 
appealing  if  two  or  more  MOEs  complement  each  other  by  assessing  differ¬ 
ent  aspects  of  the  same  operation.  Both  possibilities  are  present  in  a 
three-member  suite  consisting  of: 

•  Smuggler  success  rate 

•  Interdiction  rate 

•  Boarding  success  rate. 

The  smuggler  success  rate  assesses  force  effectiveness  in  terms  of 
impact  on  the  smuggler's  overall  operation.  The  MOE  is  figured  on  the 
AOR  taken  as  a  whole,  including  the  times  and  places  not  covered  by 
interdiction  forces;  this  latter  aspect  is  important  because  it  accounts 
for  the  opportunities  remaining  open  to  smugglers  despite  the  interdic¬ 
tion  effort.  Taking  the  smuggler's  point  of  view  also  facilitates 
dividing  the  criminal  population  according  to  level  of  sophistication, 
setting  the  stage  for  scoring  interdiction  effectiveness  against  each 
segment.  Results  in  this  disaggregated  form  should  quickly  reveal 
whether  interdiction  forces  have  concentrated  their  effort  against  the 
easiest  targets,  a  key  problem  when  interdiction  rate  and  boarding 
success  rate  are  used  alone  as  MOEs. 

The  interdiction  rate  assesses  force  effectiveness  in  terms  of  the 
fraction  of  smuggling  effort  interdicted.  In  contrast  to  the  smuggler 
success  rate,  this  MOE  is  figured  only  for  the  portion  of  the  tactical 
area  (in  time  and  space)  where  the  forces  put  smugglers  at  risk  of 
interdiction,  characterizing  how  well  the  forces  perform  against  the 
threat  within  their  reach. 

The  boarding  success  rate  assesses  force  efficiency  in  terms  of  how 
often  a  boarding  results  in  a  seizure.  It  is  a  direct  measure  of  how 
well  boarding  activity  is  targeted.  One  key  difficulty  with  this  MOE  is 
that  it  tempts  forces  to  concentrate  on  the  easy  targets  and  rely  on 
only  the  strongest  cues;  a  way  to  combat  this  bias  is  to  break  down  the 
assessment  according  to  different  types  of  boardings,  for  example: 

•  Proven  intelligence  cuing  methods 

•  Experimental  or  unproven  cuing  methods 
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•  Random  selections  and  other  methods  designed  to  detect 
smugglers  who  have  figured  out  how  to  avoid  cuing 
techniques  currently  in  use. 

It  might  also  be  necessary  to  introduce  a  measure  of  how  fully  boarding 
capacity  is  used  to  counteract  the  implicit  rewards  to  withholding 
resources  from  low  probability  endeavors. 

Taken  together,  these  three  MOEs  provide  the  means  to  evaluate  a 
single  set  of  operations  from  both  the  smuggler  and  law  enforcement 
perspectives,  simultaneously.  Further,  both  efficiency  and  effective¬ 
ness  of  the  interdiction  forces  are  assessed  jointly. 

Three  factors  limit  this  suite  of  MOEs  to  a  narrow  range  of  appli¬ 
cation.  First,  this  combination  is  meaningful  only  for  the  surface 
maritime  smuggling  threat,  the  only  realm  in  which  boarding  success  rate 
is  defined.  Second,  the  MOE  suite  can  be  used  only  in  postulated 
scenarios,  because  data  needed  to  compute  interdiction  rate  and  smuggler 
success  rate  are  not  available  from  real  operations.  Third,  even  if 
real-world  data  were  of  sufficient  quality,  interdiction  events  happen 
at  too  low  a  rate  to  stratify  the  data  according  to  smuggler  sophisti¬ 
cation  and  boarding  type;  dissecting  operations  to  this  extent  is  pos¬ 
sible  only  in  postulated  scenarios  where  event  rates  and  the  passage  of 
time  are  not  constraints  on  using  models  or  simulations  to  accumulate  a 
sufficient  mass  of  data. 

SUITE  EVALUATED  ON  MOE  CRITERIA 

The  MOE  attributes  and  evaluation  criteria  presented  in  section  2 
are  as  meaningful  for  a  combination  of  measures  as  they  are  for  individ¬ 
ual  MOEs;  indeed,  two  of  the  criteria  are  pertinent  principally  for 
suites.  Keep  in  mind  throughout  the  following  evaluation  that  the 
context  is  constrained  to  just  postulated  scenarios  of  surface  maritime 
interdiction. 

Mission  Oriented 

Yes.  The  suite  provides  a  three- tiered  evaluation:  effectiveness 
against  the  smuggler's  overall  program,  effectiveness  against  vessels  at 
risk  to  interdiction  forces,  and  efficiency  of  the  boarding  program. 
Doing  well  on  each  measure  manifestly  contributes  to  mission  success. 
There  is  no  explicit  priority  among  the  three  measures,  however;  differ¬ 
ent  resource  allocations  will  result  in  changes  in  the  profile  of  MOE 
scores,  in  effect  assigning  implicit  weights. 


1.  Much  of  the  discussion  in  this  section  still  applies  to  a  more 
restricted  combination  of  just  smuggler  success  rate  and  interdiction 
rate;  the  pair  could  be  used  in  a  broader  array  of  scenarios  than  just 
surface  maritime  interdiction. 
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Measurable 


Yes .  Analyses ,  simulations ,  and  field  exercises  have  the  advantage 
that  scenario  durations  and  event  rates  can  be  controlled  to  provide 
high-quality  data  in  desired  amounts.  One  must  remember  these  data  are 
not  real,  but  merely  the  product  of  some  model  or  simulation  that  could 
harbor  subtle  (but  critical)  omissions  and  errors. 

Quantitative 

Yes.  All  three  measures  are  numerical. 

Discriminating 

Yes,  but  possibly  in  a  misleading  way.  In  a  postulated  scenario, 
one  can  generate  enough  data  to  assure  that  the  results  are  not  handi¬ 
capped  by  small  sample  size,  but  these  data  may  reflect  conceptual 
errors  imbedded  in  the  models  of  smugglers,  interdiction  forces,  and 
their  interaction.  Distinctions  that  seem  compelling  in  a  postulated 
scenario  may  prove  totally  unreliable  in  reality  because  of  errors  and 
oversights  in  the  models. 

Realistic 

Potentially  yes.  Presumably,  a  competent  analyst  would  not 
deliberately  model  interdiction  in  an  unrealistic  manner,  but  ambigui¬ 
ties  and  information  gaps  may  produce  the  same  effect.  The  problem  is 
likely  to  be  more  critical  for  analyses  and  simulations  because  live 
exercises  involving  interdiction  forces  engage  at  least  part  of  the  real 
world. 

Objective 

Possibly,  depending  on  the  formulation  and  modeling  of  the  postu¬ 
lated  scenario.  Without  the  discipline  of  real-world  data,  biases  might 
become  embedded  in  the  design  and  execution  of  the  models. 

Appropriate 

Yes,  provided  the  models  are  tailored  to  be  representative  of  the 
forces  under  study. 

Sensitive 

Yes,  with  proper  model  design.  The  chief  risk  comes  from  concen¬ 
trating  on  internal  details  of  the  scenario  and  models  while  overlooking 
external  factors  that  end  up  driving  the  answer. 
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Promptly  Responsive 

No.  In  real-world  operations,  the  key  sources  of  feedback  delay 
are  data  reporting  and  slow  event  rates.  Though  the  passage  of  time  can 
be  accelerated  in  a  simulation,  the  elapsed  time  inside  the  scenario  to 
accumulate  a  critical  mass  of  data  is  unchanged. 

Simple  To  Understand 

Yes.  Intuitive  understanding  of  all  three  measures  is  consistent 
with  their  more  formal  meanings. 

Validated 

No.  However  obvious  and  appealing  these  measures  might  be,  they 
still  have  not  been  validated  in  any  real-world  sense.  Neither  is  it 
clear  how  validation  could  be  carried  out,  given  the  improbability  of 
collecting  real-world  data  of  the  kind  needed  to  compute  either  the 
smuggler  success  rate  or  interdiction  rate. 

Balanced 

Yes.  Though  these  three  measures  may  not  cover  everything  inter¬ 
diction  forces  do,  they  provide  a  nested  assessment  of  three  central 
issues:  effectiveness  against  the  smuggler's  overall  program,  effec¬ 

tiveness  against  smuggling  vessels  at  risk  to  Interdiction,  and  effi¬ 
ciency  of  using  a  critical  resource,  boarding  capacity. 

Compact 

Yes.  At  first  glance,  it  might  seem  that  this  suite  of  three  MOEs 
is  redundant  in  that  they  all  employ  at  least  some  of  the  same  informa¬ 
tion,  but,  in  fact,  each  measure  takes  a  different  field  of  view  and 
illuminates  an  aspect  of  the  problem  beyond  reach  of  the  other  two 
measures.  This  complementarity  suggests  that  one  could  not  achieve  the 
same  effect  with  a  smaller  set,  or  at  least  not  one  retaining  any  of 
these  three  measures. 

SUMMARY 

This  suite  of  three  illustrates  how  several  MOEs  might  be  combined 
into  a  system  of  mutually  supporting  measures .  Properties  of  one  can 
offset  deficiencies  of  another;  multiple  MOEs  can  evaluate  a  given  set 
of  operations  from  different  but  complementary  frames  of  reference, 
while,  in  the  process,  producing  a  more  balanced  and  comprehensive 
result.  On  the  other  hand,  this  combination  is  restricted  to  a  narrow 
range  of  application:  postulated  scenarios  of  surface  maritime  interdic¬ 
tion.  The  insights  obtained  through  this  trio  of  MOEs  should  be 
moderated  by  concern  for  how  faithfully  the  modeling  captures  the  real 
world,  particularly  smuggler  capabilities  and  behavior. 
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SECTION  11 


PERFORMANCE  SUMMARY  FOR  INTERDICTION  MOEs 


INTERDICTION  MOEs  APPLIED  TO  REAL  OPERATIONS 

The  tables  on  the  following  pages  briefly  summarize  the  standing  of 
the  six  MOEs  on  each  of  the  11  criteria.  Table  11-1  concerns  their 
application  in  real  operations.  All  six  MOEs  have  such  serious  draw¬ 
backs  in  this  context  that  none  of  them  is  likely  to  be  very  useful  for 
evaluating  tactical  performance  of  actual  interdiction  forces.  The 
dominant  shortcomings  include  the  following: 

•  Interdiction  rate,  smuggler  success  rate,  and  market  share 
are  not  measurable  because  they  rely  on  unknowable  infor¬ 
mation  about  the  real  flow  of  illicit  drugs. 

•  Raw  seizures,  boarding  success  rate,  and  interdiction  rate 
all  harbor  an  implicit  incentive  for  the  forces  to  concen¬ 
trate  on  the  least  sophisticated  slice  of  the  smuggler 
spectrum. 

•  Import -export  price  difference  and  market  share  both 
respond  to  a  host  of  factors  beyond  the  realm  of  interdic¬ 
tion,  making  it  difficult  to  tell  just  what  factors  might 
be  responsible  for  an  observed  change. 

•  All  six  MOEs  require  long  periods  to  give  officials  feed¬ 
back  about  the  effects  of  a  change  in  operations.  The 
import-export  price  difference  and  market  share  rely  on  a 
lengthy  collection  and  analysis  process  that  provides  data 
only  a  year  or  more  after  the  fact.  The  other  four  MOEs 
depend  on  events  that  accumulate  so  slowly  that  even  year- 
to-year  comparisons  require  extraordinary  changes  in  the 
MOE  to  recognize  that  a  real  change  has  occurred. 


INTERDICTION  MOEs  APPLIED  TO  POSTULATED  SCENARIOS 

Table  11-2  summarizes  application  of  these  MOEs  in  postulated 
scenarios  such  as  war  games,  simulations,  and  exercises;  the  table  also 
Includes  the  three -member  suite  of  MOEs  for  surface  maritime 
interdiction  operations,  as  explored  in  section  10.  Postulated 
scenarios  have  some  advantages  that  ease  selected  problems  encountered 
in  real  operations. 
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Table  11-1.  Summary  of  MOE  evaluations  for  real  interdiction  operations 
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Table  11-2.  Summary  of  MOE  evaluations  for  postulated  scenarios 
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•  Both  sides  of  the  contest  can  be  instrumented  as  desired; 
the  evaluator  has  access  to  ground  truth  about  all  simu¬ 
lated  events,  notably  the  timing,  routing,  and  quantities 
of  smuggled  contraband.  Virtually  anything  in  a  postu¬ 
lated  scenario  is  subject  to  measurement,  though  the 
result  may  be  specious  unless  great  care  is  taken  to 
assure  realistic  results. 

•  External  factors  that  obscure  the  impact  of  interdiction 
forces  can  be  tracked  and  the  results  adjusted  for  their 
effects. 

•  Problems  of  low  event  rates,  long  data- accumulation  times, 
and  small  sample  sizes  can  be  addressed  by  exercising  the 
model  at  greater  length  to  build  a  sufficient  base  of 
statistics . 

Although  these  improvements  ease  difficulties  and  resolve  some 
ambiguities,  they  do  not  solve  all  the  problems. 

•  Results  of  the  game,  model,  or  simulation  will  be  no 
better  than  the  quality  of  the  modeling  that  goes  into 
them.  If  the  representation  of  smuggler  operations  is 
seriously  flawed  or  incomplete,  expect  the  analytic 
results  to  suffer  the  same  deficiencies. 

•  Biases  embedded  in  some  MOEs  (e.g.,  incentive  to  concen¬ 
trate  on  the  least  sophisticated  smugglers)  are  still 
present;  however,  with  their  comprehensive  oversight  of 
the  simulated  events,  controllers  might  be  able  to  inves¬ 
tigate  the  practical  extent  and  significance  of  the 
biases . 

•  The  number  of  raw  seizures  still  is  poorly  conceived  as  an 
MOE,  primarily  because  it  lacks  any  reference  to  the  level 
of  effort  on  either  side.  Whatever  the  context,  the 
meaning  of  rising  or  falling  scores  cannot  be  determined 
objectively. 

SUMMARY 

It  is  a  formidable  challenge  to  devise  measures  that  send  the  right 
message  to  the  forces,  rely  on  data  that  can  be  collected  in  real  opera¬ 
tions,  and  give  a  result  with  sufficient  sensitivity  and  responsiveness 
to  be  worth  the  trouble  of  making  the  effort.  Simultaneously  consider¬ 
ing  all  these  issues  requires  a  systematic  and  disciplined  approach  of 
the  sort  used  in  this  study. 
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None  of  the  MOEs  considered  here  are  likely  to  be  of  much  value  for 
evaluating  performance  in  real  operations;  measures  that  use  good  data 
suffer  from  serious  conceptual  problems,  notably  a  lack  of  mission 
orientation;  those  that  are  soundly  related  to  the  mission  require  data 
that  cannot  be  collected  under  any  foreseeable  circumstances . 

The  suite  of  three  MOEs  (smuggler  success  rate,  interdiction  rate, 
and  boarding  success  rate)  for  surface  maritime  interdiction  is  attrac¬ 
tive  for  postulated  scenarios  as  in  games,  simulations,  and  exercises. 
Indeed,  they  complement  each  other,  measuring  both  effectiveness  and 
efficiency.  Together,  they  capture  views  from  both  sides  of  the  contest 
in  a  common  frame  of  reference. 
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SECTION  12 


RELEVANCE  TO  MILITARY  ROLES  IN  INTERDICTION 


Because  military  forces  play  only  supporting  roles,  their 
contribution  is  to  enhance  the  primary  interdiction  campaign  as  executed 
by  the  LEAs.  An  issue  confronting  DOD  is  how  to  measure  the  military 
contribution  to  the  collective  success  of  interdiction.  Efforts  to  find 
an  answer  are  confounded  by  both  the  policy  restrictions  on  military 
activity  and  the  difficulties  of  assessing  in  isolation  just  one  part  of 
a  large  undertaking  with  many  interdependent  pieces.  This  section 
explores  this  issue,  particularly  as  it  relates  to  the  MOEs  considered 
in  earlier  sections. 

MILITARY  ROLES  IN  DRUG  INTERDICTION 

The  Posse  Comitatus  Act  of  1878  (with  subsequent  amendments)  pro¬ 
hibits  the  Army  and  Air  Force  from  enforcing  a  civil  law;  this  injunc¬ 
tion  also  extends  to  the  Navy  and  Marine  Corps  as  a  matter  of  DOD 
policy.  Military  forces  are  prohibited  specifically  from  rendering  the 
following  forms  of  direct  assistance  to  LEAs; 

•  Interdiction  of  a  vehicle,  vessel,  aircraft,  or  other 
similar  activity 

•  A  search  or  seizure 

•  An  arrest,  apprehension,  stop  and  frisk,  or  similar  activ¬ 
ity 

•  Use  of  military  personnel  for  surveillance  or  pursuit  of 
individuals,  or  as  undercover  agents,  informants,  investi¬ 
gators,  or  interrogators. 

With  some  limitations,  however,  military  personnel  may  train  law 
enforcement  officers,  provide  expert  advice,  or  maintain  and  operate 
equipment  [24  and  25] . 

Until  September  1988,  the  military  role  in  drug  interdiction  was 
limited  to  providing  equipment  and  services  to  LEAs ;  DOD  forces  did  not 
initiate  or  conduct  operations  on  their  own.  The  FY  1989  National 
Defense  Authorization  Act  [26]  opened  a  new  chapter  by  assigning  DOD 
three  missions: 


1.  These  statements  are  the  most  current  versions  of  the  missions;  they 
reflect  the  slight  language  adjustments  contained  in  [27]. 
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•  "...serve  as  the  single  lead  agency  of  the  Federal  Govern¬ 
ment  for  the  detection  and  monitoring  of  aerial  and  mari¬ 
time  transit  of  illegal  drugs  into  the  United  States." 

e  "...  integrate  into  an  effective  communications  network  the 
command,  control,  communications,  and  technical  intelli¬ 
gence  assets  of  the  United  States  that  are  dedicated  (in 
whole  or  in  part)  to  the  interdiction  of  illegal  drugs 
into  the  United  States . " 

e  Drawing  on  a  special  appropriation,  fund  the  plans  of 
state  governors  for  using  the  National  Guard  (in  non- 
Federal  status)  to  conduct  interdiction  and  counter- drug 
activities  (paraphrased  for  clarity) . 

The  Secretary  of  Defense  assigned  the  detection  and  monitoring 
mission  to  the  Chairman,  Joint  Chiefs  of  Staff  (CJCS)  for  execution.  In 
turn,  tasking  to  develop  regional  concepts  of  operations  went  to  five 
commanders  in  chief  (CINCs) :  USCINCLANT,  USCINCPAC,  CINCFOR, 
USCINCSOUTH,  and  CINCNORAD.  The  first  three  established  joint  task 
forces  (JTFs,  numbered  FOUR,  FIVE,  and  SIX,  respectively)  to  plan  and 
execute  antidrug  operations  in  their  areas  of  responsibility;  the  latter 
two  CINCs  chose  to  use  their  existing  operations  centers  rather  than 
create  a  new  field  organization. 

The  result  is  a  wide  range  of  antidrug  activities  assigned  to 
military  forces: 

•  Conduct  surveillance  with  radar  and  other  sensors  from 
satellites,  aircraft,  aerostats,  ships,  and  land  sites. 

The  Coast  Guard,  Customs  Service,  and  the  Federal  Aviation 
Administration  perform  similar  functions. 

•  Coordinate  detection  and  monitoring  activities  of  all 
participating  agencies.  Relevant  defense  intelligence  is 
woven  into  tactical  operations  through  the  JTFs  and  CINC 
operations  centers.  Once  an  aerial  target  is  identified, 
intercept  control  normally  is  handed  off  to  operations 
centers  belonging  to  LEAs . 

•  Intercept  suspected  smuggling  aircraft  for  the  purpose  of 
identification  and  covert  trail.  Military  aircraft  are 
used  outside  U.S.  borders  when  Coast  Guard  and  Customs 
interceptors  are  not  available  for  the  mission. 

•  Provide  platforms  for  interdiction  operations.  Navy  ships 
carry  Coast  Guard  law  enforcement  detachments  whose 
members  conduct  all  boardings  of  suspect  vessels;  through¬ 
out  a  visit  and  search,  the  Navy  vessel  remains  under 
Coast  Guard  tactical  control  ([17]  lays  out  full  details). 
In  other  contexts,  military  aircraft  and  motor  vehicles 


12-2 


carry  law  enforcement  agents  Into  action.  These  same 
functions  also  are  performed  with  Coast  Guard  cutters, 
Customs  Service  launches  (in  coastal  areas),  and  LEA 
aircraft. 

e  Pre-empt  smuggler  employment  of  an  operating  area. 

Congress  has  directed  the  Secretary  of  Defense  to  conduct 
military  training  exercises  in  drug  interdiction  areas  to 
the  maximum  extent  possible  [27].  Smugglers  seem  reluc¬ 
tant  to  press  their  trade  through  an  area  where  forces  are 
exercising,  whether  on  land  or  at  sea;  they  choose  to 
either  stand  down  or  head  in  another  direction.  Law 
enforcement  agents  can  take  the  opportunity  to  redeploy 
for  the  duration  of  the  exercise  and  concentrate  their 
efforts  elsewhere. 

•  Train  law  enforcement  personnel.  Examples  of  training 
subjects  include:  foreign  languages,  planning  techniques, 
surveillance,  field  tactics,  communications  security,  and 
maintenance  of  military  equipment. 

•  Provide  operational  support.  The  services  loan  equipment, 
sometimes  also  sending  military  personnel  along  to  operate 
and  maintain  the  gear.  Some  items  with  obvious  applica¬ 
tion  are  night  vision  goggles,  secure  communications, 
aircraft,  and  vehicles.  Military  facilities  also  play 
support  roles  based  on  their  special  capabilities  or 
favorable  locations  (e.g.,  Howard  AFB,  Panama;  NAS 
Roosevelt  Roads ,  Puerto  Rico ;  and  NAS  Guantanamo ,  Cuba) . 

One  striking  point  about  this  list  is  that  hardly  any  part  of  it  is 
the  exclusive  province  of  military  forces.  The  LEAs  have  developed 
their  own  forces,  base  structures,  intelligence  sources,  operations 
centers,  communications  nets,  tactics,  and  almost  every  other  facet  of 
antidrug  activities.  In  addition,  virtually  everything  a  military  unit 
might  contribute  to  the  antidrug  campaign  depends  in  some  way  on  coor¬ 
dinated  action  by  an  LEA  to  realize  a  payoff.  Accordingly,  benefits 
from  military  participation  are  shared  with  other  parties;  a  determined 
effort  to  isolate  and  evaluate  the  military  contribution  on  its  own  runs 
a  serious  risk  of  losing  sight  of  the  big  picture. 

SPECIAL  PROBLEMS  OF  EVALUATING  THE  MILITARY  CONTRIBUTION 

This  brief  sketch  of  legal  limitations  and  overlapping  roles  sets 
the  stage  for  examining  the  special  problems  of  evaluating  the  military 
contribution  to  interdiction. 

Seizures  Are  Disconnected  From  Military  Roles 

The  three  MOEs  considered  in  sections  4,  5,  and  6  (raw  seizures, 
boarding  success  rate,  and  interdiction  rate)  are  seizures  in  various 
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guises.  Attempts  to  link  military  contributions  to  any  such  measures 
are  suspect  because  military  forces  lack  the  authority  and  training  to 
seize  property  or  arrest  suspects;  only  law  enforcement  agencies 
(Customs,  Coast  Guard,  Drug  Enforcement  Administration,  state  and  local 
police,  and  many  others)  may  do  so.  If,  nevertheless,  one  chooses  to 
assess  the  DOD  contribution  via  a  seizure -related  MOE,  military  perfor¬ 
mance  depends  on  the  follow-up  performance  of  associated  LEAs;  at  the 
logical  extreme,  perfect  military  execution  would  be  meaningless  if  the 
LEAs  are  not  effective  in  capitalizing  on  the  resulting  opportunities. 
Conversely,  an  improvement  in  LEA  operations  might  be  solely  responsible 
for  increases  in  the  MOEs.  Such  MOEs  totally  fail  the  criterion  for  an 
MOE  to  be  "appropriate." 

Neither  can  military  forces  reliably  demonstrate  a  connection 
between  changes  in  their  operations  and  subsequent  seizure  statistics,  a 
failure  of  the  criterion  for  an  MOE  to  be  "sensitive."  Focusing  on 
seizures  underscores  that  military  participants  serve  as  mere  accesso¬ 
ries  facilitating  actions  of  others;  military  actions  taken  alone  are 
devoid  of  any  intrinsic  value  in  this  context. 

Finally,  a  fundamental  practical  problem  with  MoE<?  built  on 
seizures  is  how  slowly  these  events  accumulate  over  time,  especially 
those  with  military  involvement.  The  small  sample  sizes  mean  that  huge 
changes  in  the  rates  are  necessary  to  reach  any  statistical  confidence 
that  there  is  a  real  change  in  performance,  the  essence  of  the  criterion 
"discriminating. " 

Allocating  Credit  Is  Subjective  and  Often  Arbitrary 

At  a  press  conference  following  the  defeat  of  Cuban  exiles  at  the 
Bay  of  Pigs,  President  John  F.  Kennedy  said,  "There  is  an  old  saying 
that  victory  has  a  hundred  fathers  and  defeat  is  an  orphan...."  [28]. 

The  truth  of  the  adage  is  apparent  when  everything  works  right  and  a 
large  load  of  drugs  is  seized.  Half  a  dozen  agencies  may  see  themselves 
as  instrumental  in  the  seizure,  but  claims  of  credit  may  depend  as  much 
on  fortune  and  administrative  practice  as  on  operational 
effectiveness.  Authorities  directing  multi-agency  operations  often  have 
a  choice  of  which  agency  or  unit  receives  the  nod  to  intercept 
particular  targets.  It  is  distinctly  unsatisfying  to  award  credit 
without  distinguishing  between  those  who  were  instrumental  and  those  who 
merely  happened  to  be  present;  yet,  in  the  midst  of  all  the  many 
participants,  it  is  often  difficult  to  tell  the  difference.  As  a 
result,  an  agency  or  a  service  might  see  rewards  in  leaving  its 
fingerprints  on  as  many  interdiction  events  as  possible,  even  if  its 
real  contribution  is  insignificant. 

The  smuggler  success  rate  is  no  better  basis  for  separately 
measuring  a  military  contribution.  First,  a  seizure  is  the  most  promi¬ 
nent  way  a  smuggling  mission  can  fail,  already  described  as  an  inappro¬ 
priate  criterion  for  DOD  forces.  Second,  though  this  MOE  is  a  natural 
gauge  of  interdiction's  effect  on  smugglers,  the  link  to  performance  of 
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individual  agencies  is  indirect  and  hard  to  trace.  The  measure  recog¬ 
nizes  effects  other  than  seizures  and  takes  into  account  opportunities 
for  smugglers  to  go  around  interdiction  forces,  but  these  virtues  are 
realized  only  in  the  aggregate.  Unless  one  agency  has  the  exclusive 
franchise  in  a  sizeable  area,  its  contribution  will  be  described  as 
contingent  on  the  related  performance  of  other  agencies. 

Looking  at  the  other  side  of  the  struggle,  the  smugglers  themselves 
can  and  do  redirect  the  drug  flow  among  various  modes  and  routes,  while, 
in  the  process,  changing  the  opportunities  for  units,  agencies,  and 
geographic  regions.  As  a  result,  the  smuggler  success  rate  and  the 
three  seizure-based  MOEs  could  appear  to  make  strong  moves  (in  either 
direction)  without  any  real  change  in  performance  by  the  interdiction 
forces.  The  more  finely  one  tries  to  break  down  operations,  the  greater 
the  possibility  of  this  effect;  all  credibility  evaporates  by  the  time 
one  attempts  to  allocate  credit  among  agencies  in  a  relatively  small 
AOR. 


Such  problems  for  seizure -related  MOEs  are  elementary  when  compared 
to  allocating  credit  under  the  other  two  MOEs:  import-export  price 
difference  and  market  share.  Both  measures  embody  a  wide  range  of 
effects,  only  some  of  which  are  due  primarily  to  the  action  of  interdic¬ 
tion  forces  in  the  AOR  under  study.  These  MOEs  are  broadly  inclusive, 
reflecting  the  balance  of  opportunity  over  all  smuggling  alternatives 
and  the  effects  of  law  enforcement  activity  well  beyond  the  scope  of 
interdiction.  An  improvement  in  one  of  these  MOEs  might  signal  a  local 
success,  but,  in  fact,  the  change  might  just  as  well  stem  from  altered 
risks  and  rewards  in  regions  far  beyond  the  AOR  of  interest. 
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SECTION  13 


SUMHARY  AND  CONCLUSIONS 


Peter  Reuter  and  colleagues  [18]  devised  methods  of  assessing  the 
policy  question  of  whether  interdiction  could  make  illicit  drugs  any 
less  available  in  the  United  States  and  generally  concluded  the  pros¬ 
pects  were  poor.  This  study  takes  a  narrower  focus,  accepting  as  given 
that  some  resources  will  be  devoted  to  interdiction  and  then  seeking 
methods  of  measuring  operational  performance;  the  results  show  that  the 
task  has  no  easy  solutions . 

The  main  quest  was  to  find  ways  of  measuring  how  well  interdiction 
forces  are  performing,  preliminary  to  assessing  how  military  forces  are 
contributing  to  the  success  of  interdiction.  When  used  to  assess  real 
operations,  all  six  of  the  candidate  MOEs  proved  to  have  one  or  more  of 
these  critical  deficiencies: 

•  Require  data  that  cannot  be  collected  in  real  operations 

•  Offer  incentives  to  conduct  operations  in  ways  that 
actually  undermine  the  mission 

•  Respond  strongly  to  events  that  have  nothing  to  do  with 
the  effectiveness  of  the  forces  under  evaluation 

•  Are  beset  by  such  low  event  rates  or  high  statistical 
variability  that  real  changes  in  performance  may  be  recog¬ 
nized  only  when  the  MOE  moves  in  implausibly  large  steps. 

Furthermore,  none  of  these  impediments  has  high  prospects  of  being 
overcome  in  the  foreseeable  future . 

The  military  role  in  interdiction  is  limited  by  statute  and  policy 
to  supporting  activities.  Lacking  the  powers  of  search,  seizure,  and 
arrest,  realizing  benefit  from  military  support  to  interdiction  almost 
always  depends  on  final  action  by  a  law  enforcement  agent.  In  addition, 
though  DOD  forces  serve  in  a  wide  variety  of  roles,  many  of  them 
parallel  similar  functions  performed  by  LEAs.  These  restrictions  and 
the  lack  of  uniqueness  combine  to  make  it  all  but  impossible  to  cleanly 
separate  military  roles  and  judge  their  value  on  their  own  terms. 

Nevertheless,  the  question  of  characterizing  the  effectiveness  of 
the  military  contribution  to  antidrug  operations  still  remains .  One 
possibility  is  to  focus  narrowly  on  the  performance  of  actual  military 
activities.  For  example,  the  mission  of  detection  and  monitoring  might 
be  characterized  in  terms  of  coverage,  probability  of  detection, 
timeliness  of  notification  to  LEAs,  fraction  of  targets  tracked  to 
intercept,  and  similar  technical  criteria.  Such  an  approach  has  two 
attractive  advantages:  (1)  the  necessary  data  are  internal  to  military 


operations,  not  requiring  collection  from  the  smugglers  and  LEAs,  and 
(2)  the  measures  are  of  a  kind  familiar  to  military  forces  in  other 
operational  settings.  There  is  a  down  side,  however:  doing  well  on  such 
technical  criteria  gives  no  assurance  that  the  effort  has  any  useful 
impact.  Smugglers  still  may  find  enough  opportunities  to  evade, 
deceive,  or  circumvent  so  they  can  maintain  the  flow  of  drugs.  As  an 
extreme,  envision  an  intensive  and  expensive  effort  to  attain  high 
technical  performance,  only  to  learn  later  that  drug  purveyors  had 
outflanked  the  campaign  through  a  shift  of  methods. 

On  the  other  hand,  some  of  the  MOEs  examined  in  this  study- - 
particularly  the  trio  discussed  in  section  10- -appear  to  be  attractive 
for  assessments  in  postulated  scenarios  such  as  games,  simulations,  or 
exercises.  The  roles  of  all  participants  can  be  modeled  and  event  data 
collected  to  any  desired  extent.  In  such  applications,  the  emphasis 
shifts  from  collecting  evidence  on  operations  to  modeling  those  opera¬ 
tions  in  a  sufficiently  realistic  manner.  If  the  models  err  too  far, 
the  results  will  be  not  only  wrong  in  detail,  but  also  misleading  about 
their  implications  for  real  operations.  Validation  is  sure  to  remain  a 
continuing  problem  in  most  such  applications.  Nevertheless,  these 
postulated  contexts  may  offer  the  best  chance  of  gaining  substantial 
insight  into  how  military  forces  may  be  used  to  best  effect  while 
supporting  drug  interdiction. 
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MOEs:  A  GENERAL  DISCUSSION 


WHAT  IS  A  MEASURE  OF  EFFECTIVENESS? 

Most  people  would  assert  that  they  understand  the  meaning  of  the 
term  "measure  of  effectiveness"  (MOE) .  Miles  per  gallon,  rate  of  return 
on  investment,  casualty  rate,  and  probability  of  detection  are  just  a 
few  examples  of  MOEs  from  a  variety  of  settings.  The  literature  of 
operations  research  includes  many  definitions  of  MOEs,  usually  reflec¬ 
ting  some  specific  application  the  author  has  in  mind.  In  this  work, 
the  motivation  for  developing  MOEs  is  to  support  officials  in  the  mili¬ 
tary  and  LEA  chains  of  command  as  they  assign  roles  and  missions,  allo¬ 
cate  resources,  and  assess  mission  performance.  The  orientation  is 
clearly  toward  decision-making  in  an  operational  context.  With  that 
proviso,  the  following  definition  is  offered  as  a  fundamental  concept: 

An  MOE  is  a  quantitative  criterion  for  making  judg¬ 
ments  or  decisions.  In  general,  MOEs  summarize 

performance  of  a  system  (whether  physical  or 

organizational)  in  comparison  with: 

•  A  standard,  a  goal,  or  an  ideal- -the  perfect  system 

•  Some  alternative  system,  real  or  conceptual 

•  The  same  system  in  another  time  or  context. 

MOEs  may  be  put  to  a  number  of  different  uses: 

•  Score  operational  performance  of  systems 

•  Determine  how  much  of  a  specific  resource  is  necessary  to 
satisfy  a  goal 

•  Compare  mission  effectiveness  achieved  with  alternative 
resource  packages 

•  Measure  changes  in  performance  resulting  from  altered 
input  factors  (forces,  equipment,  tactics,  or  rules  of 
engagement) 

•  Detect  an  unexpected  change  in  operational  effectiveness, 
perhaps  as  a  cue  for  finding  the  cause. 

WHY  MIGHT  MEASURES  OF  EFFECTIVENESS  BE  NEEDED? 

A  skillfully  constructed  suite  of  MOEs  can  serve  a  host  of  useful 
purposes.  At  the  policy  level,  the  MOEs  themselves  concisely  summarize 
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what  it  means  to  succeed  in  the  mission.  At  the  operational  level, 
numerical  valuations  of  the  MOEs  might  capture  performance  of  selected 
forces  in  a  particular  time  frame  and  area  of  operations. 

Lacking  MOEs,  performance  reports  tend  to  become  an  amalgam  of 
figures  on  resource  expenditures,  anecdotes  of  specific  operations,  and 
carefully  tailored  contrasts  of  current  versus  previous  operations. 
Comparisons  of  performance  from  one  period  to  the  next  are  liable  to 
lack  credibility  in  the  absence  of  a  consistent  program  constructed 
around  a  stable  set  of  criteria. 

MOEs  assist  in  recognizing  activities  that  previously  were  success¬ 
ful  but  have  since  lost  their  relevance;  officials  in  the  field  can 
curtail  operations  that  have  not  met  expectations  so  that  the  resources 
may  go  where  greater  benefits  are  anticipated.  MOEs  that  measure  mis¬ 
sion  accomplishment  also  aid  in  evaluating  new  concepts  of  operation 
(CONOPS) ,  and  routine  use  of  MOEs  helps  to  focus  attention  on  discover¬ 
ing  why  some  kinds  of  operations  are  successful  and  others  are  not. 

If  MOEs  can  be  tied  to  costs ,  they  become  a  tool  to  help  allocate 
resources.  If  the  budget  Is  reduced,  where  will  the  cuts  have  the 
smallest  impact  on  performance?  If  additional  funds  become  available, 
where  should  they  be  applied  for  best  effect?  Is  the  current  program  in 
balance?  Would  results  stand  to  improve  through  shifting  some 
resources? 

Whatever  the  level  of  leadership,  a  suite  of  well-conceived  MOEs 
clarifies  the  purpose  of  operations,  serving  as  a  shorthand  statement  of 
operational  goals.  Subordinate  officials  know  the  specific  criteria  on 
which  they  expect  to  be  evaluated;  in  turn,  they  naturally  organize 
their  operations  to  score  as  well  as  they  can  on  the  MOEs. 

Finally,  MOEs  and  the  supporting  scheme  of  analysis  are  central  to 
designing  the  data-collection  system.  Although  many  of  the  data  items 
appropriate  to  evaluating  mission  performance  are  fairly  obvious  to  a 
perceptive  observer,  a  largely  intuitive  approach  can  never  answer  any 
of  the  following  questions  adequately: 

•  Are  all  required  data  being  collected? 

•  Are  some  of  the  reported  data  superfluous? 

•  Are  the  required  data  being  collected  with  the  accuracy 
needed  to  support  the  Intended  analysis? 

•  Are  resources  being  wasted  through  collecting  data  with 
more  precision  than  needed  in  the  analysis? 

•  Are  all  participants  reporting  uniformly?  Do  their  simi¬ 
lar-appearing  data  streams  mask  important  differences  that 
dominate  the  apparent  outcomes? 
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CONCEPT  OF  A  PERFORMANCE  ASSESSMENT  SYSTEM 


Putting  the  discussion  in  terms  of  MOEs  should  be  seen  as  merely  a 
concise  way  of  talking  about  the  overall  performance  assessment  system. 
MOEs  should  be  driven  by  program  goals ,  supported  by  a  solid  analytic 
infrastructure,  and  fed  by  a  well-designed  stream  of  operational  data. 
Assessing  effectiveness  of  antidrug  operations  entails  much  more  than 
merely  promulgating  MOEs  as  technical  standards  and  then  pushing  opera¬ 
tional  data  through  formulas  to  produce  an  answer. 

Clearly  stated  goals  are  the  bedrock  on  which  the  whole  system  is 
built.  If  objectives  are  well  crafted,  formulating  MOEs  is  more  or  less 
routine,  but  if  the  policy-makers  are  not  agreed  on  what  it  means  to 
succeed  and  how  to  recognize  success  when  it  happens,  operations  ana¬ 
lysts  lack  the  context  for  developing  an  appropriate  suite  of  MOEs.  At 
best  they  can  offer  options  based  on  statements  of  possible  goals  in  an 
effort  to  elicit  intentions  from  top  officials. 

Analysis  methods  must  be  developed  to  fit  the  problem,  then  tested 
and  validated  with  real  operations.  MOEs  are  just  the  most  visible  part 
of  the  overall  analysis  structure. 

A  key  analytic  task  is  to  demonstrate  the  nature  and  strength  of 
the  link  between  goal  satisfaction  and  the  response  behavior  of  proposed 
MOEs.  If  an  MOE  moves  up  or  down,  what  does  that  result  say  about 
fulfillment  of  the  goal?  Are  there  some  conditions  where  the  answer  is 
less  definitive  or  valid  than  others? 

Another  analytic  task  is  to  investigate  the  statistical  properties 
of  candidate  MOEs.  Rare  indeed  is  the  measure  that  can  be  computed  with 
certainty;  a  number  reported  for  a  given  period  is,  in  fact,  merely  an 
estimate  compiled  with  some  error.  The  associated  range  of  uncertainty 
for  that  estimate  is  an  essential  ingredient  for  judging  whether  a 
change  from  one  period  to  another  is  significant,  or  merely  random 
variation.  Just  this  sort  of  problem  is  a  prominent  difficulty  with 
several  MOE  candidates  considered  in  the  text  of  this  memorandum. 

Reliable  data  are  essential  to  realizing  any  value  from  a  suite  of 
MOEs.  Quality  control  can  reduce  the  incidence  of  oversights  and  admin¬ 
istrative  errors,  but  real  success  in  this  regard  relies  on  motivated 
people  who  understand  and  support  what  they  are  doing. 
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CONTRASTS  TO  CLARIFY  DISCUSSIONS  OF  MOEs 

Misunderstandings  frequently  arise  in  discussions  of  MOEs  because 
the  participants  hold  unstated  assumptions.  Three  particular  sets  of 
contrasts  are  encountered  frequently  enough  to  warrant  discussion: 

e  Effectiveness,  efficiency,  and  activity 
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•  Strategic  versus  tactical  perspective 

•  Postulated  scenarios  versus  real  operations. 


Effectiveness,  Efficiency,  and  Activity 

Even  though  evaluation  criteria  routinely  are  called  measures  of 
effectiveness,  many  proposals  really  deserve  some  other  label.  Three 
distinct  varieties  arise  often  in  discussions  of  antidrug  operations: 

Effectiveness  measures  relate  the  results  of  antidrug  operations  to 
the  smuggler's  level  of  effort.  If  the  results  are  about  the  same 
as  last  year,  but  the  smugglers  moved  twice  the  volume  of  drugs, 
antidrug  forces  might  be  judged  only  half  as  effective  as  before. 
Clearly,  emphasis  is  on  assessing  the  degree  to  which  operations 
affect  the  smuggler's  success. 

Efficiency  measures  relate  operational  results  to  the  level  of 
effort  exerted  by  the  antidrug  forces.  If  operations  achieve  about 
the  same  outcome  this  year  as  last  but  use  twice  as  many  resources , 
they  might  be  judged  only  half  as  efficient  as  before.  The  termi¬ 
nology  becomes  "cost  effectiveness"  when  the  antidrug  forces’  level 
of  effort  is  expressed  in  terms  of  dollars  or  other  resources.  Two 
different  force  structures  might  achieve  the  same  results  and  so  be 
judged  equally  effective,  but  if  one  does  the  job  at  lower  cost 
than  the  other,  it  is  the  more  efficient  or  cost  effective. 

Activity  measures  describe  the  magnitude  or  intensity  of  opera¬ 
tions.  Tabulations  of  critical  events  (e.g.,  radar  contacts, 
intercepts  attempted,  arrests,  quantities  of  drugs  seized)  do  no 
more  than  indicate  the  scale  of  activity.  When  accompanied  by  a 
relevant  baseline  standard  of  comparison,  an  activity  measure  might 
be  transformed  into  a  measure  of  either  effectiveness  or 
efficiency. 


Strategic  Versus  Tactical  Perspectives 

When  discussing  antidrug  MOEs  in  any  sizeable  group,  the  partici¬ 
pants'  frames  of  reference  range  over  a  full  spectrum  from  the  strategic 
impact  on  the  national  campaign  to  fine -grain  assessment  of  tactical 
execution;  rarely  does  a  single  MOE  perform  well  across  the  full 
spectrum. 

At  the  strategic  end,  the  questions  are  large  in  scope.  Can  inter¬ 
diction  have  a  beneficial  effect  on  national  level  goals?  How  large  a 
part  in  the  overall  campaign  should  interdiction  play?  References  [ A- 1 ] 
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and  [A-2]  are  good  examples  of  large-scale  inquiries;  incidentally,  they 
conclude  that  interdiction  has  little  prospect  of  affecting  price, 
purity,  or  availability  of  illicit  drugs  at  the  retail  level. 

At  the  tactical  end,  there  is  a  presumption  that  interdiction 
supports  national  goals,  even  though  the  relationship  might  not  be 
quantified.  As  a  result,  the  questions  are  somewhat  narrower.  Which  of 
several  candidate  CONOPS  stand  to  produce  the  best  results?  If  addi¬ 
tional  resources  become  available ,  how  should  they  be  applied?  How 
effectively  (or  efficiently)  are  the  forces  executing  current 
operations? 

This  study  concentrates  on  evaluating  how  well  forces  are  perform¬ 
ing  and  so  tends  toward  the  tactical  tend  of  the  spectrum;  however,  the 
perspective  is  that  of  a  high  level  of  authority:  cabinet  secretaries 
and  their  subordinates  who  head  operating  agencies. 


Postulated  Scenarios  Versus  Real  Operations 

Though  the  primary  quest  is  for  MOEs  suited  to  evaluate  performance 
of  actual  forces  in  real  operations ,  the  most  common  use  might  come  in 
postulated  scenarios  such  as  planning  efforts,  war  games,  simulations, 
and  exercises. 

Postulated  scenarios  characteristically  are  under  the  full  control 
of  the  analyst,  planner,  or  exercise  director;  they  might  be  created  and 
fine-tuned  to  match  the  current  understanding  of  smuggler  activities  or 
varied  to  explore  effects  of  uncertainties  and  future  possibilities.  In 
either  case,  the  magnitude,  character,  and  location  of  the  threat  is  an 
input  to  the  evaluation.  In  addition,  the  effect  of  interdiction  forces 
on  this  specific  smuggling  threat  is  determined  through  models  of  the 
process  or,  in  the  case  of  field  exercises,  measures  under  controlled 
conditions.  Appendix  C  goes  into  more  detail  on  using  MOEs  in  this  way. 

Real  operations ,  in  contrast,  are  characterized  by  a  threat  that 
goes  out  of  its  way  to  remain  unnoticed  and  difficult  to  quantify. 
Estimates  of  threat  activity  are  partial  at  best  and  entail  important 
uncertainties.  Data  concerning  the  origin,  amount,  and  timing  of  drug 
smuggling  are  hard  to  acquire,  leaving  some  attractive  MOEs  impossible 
to  apply. 

Candidate  MOEs  are  evaluated  in  the  main  text  explicitly  for  their 
usefulness  in  both  postulated  scenarios  and  real  operations. 
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The  Atlantic  Fleet  routinely  participates  in  Caribbean  antidrug 
operations.  In  addition  tc  making  a  large  contribution  to  surface  and 
airborne  surveillance,  ships  of  the  Fleet  carry  Coast  Guard  law 
enforcement  detachments  (LEDETs)  whose  members  board  suspect  vessels, 
conduct  searches,  seize  contraband,  and  arrest  suspects  [B-l].  These 
seizures  often  become  the  focus  of  discussions  concerning  whether 
military  forces  are  being  effective  in  drug  interdiction  operations. 

This  appendix  uses  the  principles  of  probability  and  statistics  to 
assess  the  usefulness  of  seizure  data  expressed  as  the  boarding  success 
rate.  It  begins  with  a  review  of  some  actual  results  and  then  proceeds 
to  examine  several  relevant  issues  about  this  MOE.  The  presentation  is 
structured  deliberately  as  a  tutorial  to  make  the  application  of 
statistical  methods  as  clear  as  possible. 

HISTORIC  DATA 

Records  of  past  operations  suffer  from  a  number  of  important 
problems,  including  reporting  gaps  and  inadequate  definitions.  These 
difficulties  aside,  summary  results  from  1988  illustrate  the  nature  and 
magnitude  of  the  numbers  involved. 

Because  USN  and  USCG  ships  were  not  allocated  in  sufficient  numbers 
to  maintain  a  continuous  Caribbean  patrol ,  operations  occurred  in  pulses 
of  varying  size  and  duration.  During  1988,  COMCARIBRON1  conducted  12 
pulses,  the  results  of  which  are  tabulated  in  table  B-l.  Highlights 
from  these  operations  include: 

•  5  of  12  pulses  produced  no  seizures 

•  313  vessels  were  boarded,  largely  for  one  of  two  reasons: 

Specific  prior  intelligence  on  the  vessel 
Appearance  or  behavior  fitting  a  "profile"  of  drug 
smugglers 

•  54  of  the  boarded  vessels  were  positive  for  drug  history 
in  the  El  Paso  Intelligence  Center  (EPIC)  data  base 


1.  The  Commander,  Caribbean  Squadron,  is  a  Coast  Guard  officer 
subordinate  to  Coast  Guard  District  Seven  headquartered  in  Miami.  He 
exercised  tactical  control  of  USCG  cutters  and  USN  ships  dedicated  to 
antidrug  operations  in  the  Caribbean  during  1988. 


•  9  vessels  were  found  to  have  significant  amounts  of 
marijuana  on  board,  totalling  36,850  pounds 

•  2  more  vessels  showed  traces  of  marijuana  residue 

•  1  more  vessel  was  boarded  on  the  basis  of  prior 
intelligence;  22  hours  of  search  failed  to  reveal  any 
drugs ;  vessel  was  towed  to  port  for  continued  search 
ashore  where  master  finally  confessed  to  165  pounds  of 
cocaine  hidden  in  the  keel. 

•  Prior  intelligence  identified  all  12  seized  vessels  as 
probably  on  drug  missions  (and  so  all  were  positive  in  the 
EPIC  data  base) . 


Table  B-l.  USN/USCG  dedicated  surface  operations  in 
Caribbean- -1988 


Pulse  dates 

Ship 

days 

Vessels 

boarded 

EPIC3 

positive 

MJb 

Cc 

1/23  -  2/2 

84 

10 

0 

0 

0 

3/4  -  3/19 

82 

49 

5 

0 

0 

4/14  -  4/18 

4 

4 

0 

0 

0 

5/9  -  5/26 

82 

35 

4 

1 

0 

6/3  -  6/20 

19 

18 

8 

1 

0 

6/22  -  7/12 

20 

27 

3 

0 

0 

7/6  -  7/27 

38 

14 

8 

1 

0 

8/12  -  8/31 

75 

35 

8 

3 

0 

9/6  -  9/20 

26 

13 

4 

1 

0 

9/16  -  9/26 

25 

9 

1 

0 

0 

10/8  -  11/14 

182 

60 

4 

2 

1 

11/15  -  12/20 

121 

-12 

_9 

_2 

0 

810 

313 

54 

11 

1 

NOTE :  Commander 

Robert 

L.  Meyer,  USNR, 

compiled  the 

data  in 

this  table  from  operational  reports  while  on  an  active  duty 
training  period  at  CINCLANTFLT  during  January  1989. 

a.  El  Paso  Intelligence  Center. 

b.  Marijuana. 

c .  Cocaine . 

Boarding  success  rates  for  individual  pulses  range  from  0  to 
8.6  percent.  If  all  12  seizures  are  taken  as  successes,  the  overall 
boarding  success  rate  in  1988  is  12/313  -  0.038,  or  3.8  percent  of  all 
boardings . 
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Some  would  exclude  the  two  cases  involving  only  marijuana  residue 
because  the  drug  smugglers  had  already  completed  their  mission. 

Likewise,  principal  credit  for  finding  cocaine  in  the  keel  went  to  the 
agency  searching  ashore.  This  more  conservative  view  would  recognize 
only  9  seizures  for  an  overall  rate  of  9/313  -  0.029,  or  2.9  percent  of 
all  boardings . 

UNCONTROLLED  FACTORS 

The  small  number  of  seizures  in  a  full  year  suggests  it  will  be 
necessary  to  accumulate  data  over  long  periods  to  achieve  desirable 
statistical  properties,  notably  tight  error  bounds.  Yet,  combining 
statistics  from  many  successive  operations  raises  important  questions  as 
to  whether  the  latest  data  are  comparable  to  the  earliest.  Several 
uncontrolled  factors  can  affect  apparent  boarding  success  rates: 

•  Fraction  of  population  eligible  for  boarding  that  is 
actually  carrying  drugs.  Opportunities  for  success  are 
bounded  by  the  numbers  of  actual  drug  runners.  No  one 
knows  how  many  smugglers  are  at  sea  during  a  pulse; 
however,  there  is  good  evidence  that  the  more 
sophisticated  organizations  monitor  movements  of 
interdiction  forces  and  restrict  activities  where  risks 
are  temporarily  increased. 

•  Effectiveness  of  intelligence  cues  in  identifying  subsets 
of  the  shipping  population  as  probable  drug  carriers. 
Timely  notification  is  critical  so  tactical  forces  can 
operate  on  the  information  before  it  becomes  history. 
Profiles  of  smuggling  vessels  help  to  sort  li cely  targets 
from  a  vast  background  population;  too  much  reliance  on 
them,  however,  can  distract  attention  from  sophisticated 
smugglers  who  discern  the  profile  and  adjust  their 
operations  to  avoid  it. 

•  Sophistication  of  smuggler  countermeasures.  Smugglers  are 
much  less  blatant  than  in  the  past.  No  longer  do  they 
stack  marijuana  bales  on  deck  in  plain  view,  but  now  their 
countermeasures  go  further  to  include  decoys,  deception, 
and  encrypted  communications .  This  arena  is  dynamic  with 
no  clear  limits  given  the  vast  amounts  of  money  available 
to  buy  talent  and  equipment. 

•  Effectiveness  of  smuggler  methods  for  hiding  drugs  aboard 
ship.  Hidden  compartments  and  towed  capsules  are  just  two 
of  the  measures  making  searches  long  and  difficult. 

•  Search  effectiveness  of  boarding  parties.  Training, 
skill,  morale,  determination,  and  alertness  are  some  of 


the  determinants  of  boarding-party  effectiveness .  As 
smugglers  become  better  at  hiding  the  goods,  searches 
become  longer,  more  intrusive,  and  more  exhausting. 


Any  real-world  attempt  to  assess  the  impact  of  changes  in  operational 
procedures  and  tactics  must  be  sensitive  to  the  possibility  that  an 
observed  change  could  be  due  to  variations  in  one  or  more  of  these 
factors . 

ANALYSIS 

Two  questions  arise  repeatedly: 

•  Why  do  so  many  pulses  fail  to  produce  any  seizures?  This 
question  is  a  serious  issue;  as  in  any  endeavor,  morale  of 
the  forces  involved  responds  directly  to  perceptions  of 
success  and  failure. 

•  If  operational  changes  are  introduced,  how  can  authorities 
judge  whether  i  apparent  change  in  results  is  real  or 
just  a  fluke? 

Each  question  is  restated  in  a  formal  manner  and  then  examined  in  the 
setting  of  the  1988  statistics  from  Caribbean  operations. 

Question  1 

Of  vessels  meeting  criteria  for  boarding,  a  proportion  p  carry 
significant  amounts  of  drugs.  What  is  the  probability  that  n 
boardings  will  reveal  0  vessels  carrying  drugs? 

ANSWER:  The  question  parallels  the  classic  probability  problem  of 
drawing  black  and  white  balls  from  an  urn  without  replacement  (would  not 
revisit  i  vessel  just  boarded) .  The  hypergeometric  probability 
distribution  precisely  describes  this  process,  but,  for  large  numbers  of 
boardings,  the  much  handier  binomial  distribution  gives  practically  the 
same  results. 

The  binomial  distribution  has  the  following  form: 
n  -  number  of  boardings 
x  -  number  of  seizures 


p  -  x/n  -  boarding  success  rate 
p(x;  n,  p)  -  (”)  pX  ( 1  -  p)n  x  -  probability  of  x  successes 


n! 


x!(n  -  x) ! 


x.,  .n-x 

p  (1  -  p) 
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np  -  mean  of  x 


J  np(l  -  p)  -  standard  deviation  of  x 

P(0;n,p)  -  (1  -  p)n  -  probability  of  no  successes  in  n  boardings. 

Table  B-2  presents  selected  values  of  the  binomial  distribution; 
underlined  entries  are  used  in  the  sample  applications  following  the 
table . 


Table  B-2.  Selected  values  of  binomial  distribution 


p 

1  -  p 

n 

P(0) 

1  -  P(0) 

Mean 

np 

Standard 
deviation 
\Jnp(l  -  p) 

0.01 

0.99 

10 

0.90 

0.10 

0.10 

0.31 

30 

0.74 

0.26 

0.30 

0.55 

aoo 

0.37 

0.63 

1.00 

0.99 

300 

0.05 

0.95 

3.00 

2.97 

1,000 

0.000043 

0.999957 

10.00 

3.15 

0.03 

0.97 

10 

0.74 

0^16 

0.30 

0.54 

30 

0.40 

0.60 

0.90 

0.93 

100 

0.05 

0.95 

3.00 

1.71 

300 

0.0001 

0.9999 

9.00 

8.73 

0.05 

0.95 

10 

0.60 

0.40 

0.50 

0.69 

30 

0.21 

0.79 

1.50 

1.19 

100 

0.006 

0.994 

5.00 

2.18 

300 

0 . 0000002 

0.9999998 

15.00 

3.77 

0.10 

0.90 

10 

0.35 

0.65 

1.00 

0.95 

30 

0.04 

0.96 

3.00 

1.64 

100 

0.00003 

0.99997 

10.00 

3.00 

Sample  Applications  of  the  Binomial  Probability  Table 

If  3  percent  of  vessels  meeting  boarding  criteria  are  carrying 
drugs,  there  is  only  a  26 -percent  chance  of  one  or  more  seizures  in 
10  boardings.  In  1988,  six  pulses  each  resulted  in  about  10  boardings 
(the  range  is  4  to  18);  three  of  these  pulses  resulted  in  a  seizure, 
almost  double  the  expected  26  percent.  Continuing  at  a  3 -percent  rate, 

«  30  boardings  have  a  60-percent  chance  of  at  least  one  success.  The 

12  pulses  in  1988  averaged  26  boardings;  7  of  them  resulted  in  seizures, 
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virtually  the  same  as  60  percent  (7/12  -  58  percent).  Clearly,  the 
number  of  pulses  coming  up  empty-handed  is  consistent  with  the  overall 
seizure  rate. 

If  changes  in  tactics  or  operational  procedures  tighten  boarding 
criteria  so  as  to  raise  the  density  of  drug  smugglers  to  5 -percent  of 
all  boardings,  a  pulse  of  10  boardings  has  a  40-percent  chance  of  at 
least  one  success.  Pulses  of  30  boardings  have  a  79-percent  chance  of 
finding  one  or  more  drug  smugglers. 

If  these  changes  are  so  selective  that  the  proportion  carrying 
drugs  rises  to  1  in  10  (p  -  0.1),  then  30  boardings  have  a  96-percent 
chance  of  revealing  at  least  one  smuggler.  On  the  average,  30  boardings 
under  such  conditions  should  turn  up  3  seizures. 

Finally,  high  levels  of  success  might  prompt  drug  smugglers  to 
transfer  their  trade  to  alternate  routes  and  so  drive  the  proportion  of 
smugglers  down,  for  example  to  1  in  100  (p  -  0.01)  boardings.  Then,  a 
pulse  of  30  boardings  would  have  a  74-percent  chance  of  coming  up  empty- 
handed  (or  just  a  26-percent  chance  of  one  or  more  seizures) . 

Question  2 

Suppose  intelligence  methods  and  operational  tactics  are  revised  to 
concentrate  boardings  on  vessels  having  a  high  probability  of  being  drug 
manners.  How  much  must  the  boarding  success  rate  improve  before  being 
confident  that  the  difference  is  statistically  significant,  not  merely  a 
chance  result? 

ANSWER :  This  question  calls  for  comparing  boarding  success  rates 
seen  before  and  after  introducing  the  revised  methods  and  tactics.  A 
large  but  necessary  assumption  is  that  all  uncontrolled  factors  are 
substantially  the  same  in  the  new  set  of  boardings  as  in  the  comparable 
historic  period.  If  not,  the  action  of  these  other  factors  might  either 
mask  a  true  difference  or  falsely  suggest  that  the  new  methods  have  made 
a  difference. 

Let  the  following  variables  describe  the  new  sample: 

n  —  number  of  boardings  in  new  sample 

m  -  number  of  .uccesses  in  new  sample 

p  -  sample  proportion  of  successes,  m/n. 

The  sample  proportion  p  is  an  estimate  of  the  fraction  of  drug 
smugglers  in  the  class  of  vessels  that  meets  boarding  criteria.  The 
sample  actually  selected  for  boarding  may  have  a  greater  or  lesser 
-'»*«T>ortion  of  smugglers  than  the  complete  population  satisfying  the 
i.ng  criteria. 


The  method  for  answering  this  question  calls  on  using  a  statistical 
device  known  as  the  sampling  distribution.  Imagine  that  a  large  number 
of  independent  samples  (of  size  n)  could  be  collected  from  the  historic 
period.  If  all  the  sample  proportions  were  arrayed  in  sequence  from 
smallest  to  largest,  the  result  would  be  a  sampling  distribution1 
something  like  the  one  shown  in  figure  B-l.  If  there  were  enough  such 
samples,  the  average  proportion  would  converge  to  the  true  historic 
value  P.  The  proportion  computed  from  a  new  sample  (such  as  one  taken 
after  some  change  in  operations)  could  be  compared  to  the  sampling 
distribution  for  an  assessment  of  whether  there  is  a  difference. 


Figure  B-1.  Hypothetical  distribution  of  sample  proportion 


Because  the  distribution  Includes  all  possible  outcomes,  the  total 
area  under  the  curve  sums  to  probability  one,  or  100  percent.  For  any 
sample  proportion  Ps,  this  area  splits  into  two  pieces;  the  "tail"  to 
the  right  is  just  the  probability  that  a  random  sample  drawn  from 
historic  experience  would  result  in  a  proportion  larger  than  Pg.  This 
probability  is  the  risk  of  being  wrong  if  one  chooses  to  reject  the  null 
hypothesis  that  the  new  sample  is  not  different  from  the  historic 
experience,  concluding  that  the  apparent  difference  between  P  and  Ps 
stems  from  variability  in  sampling.  Common  practice  is  to  require  the 
probability  in  the  tail  (called  the  significance  level)  to  be  no  more 
than  5  percent.  With  these  preliminaries,  question  2  can  be  restated 
as : 


* 


For  a  sample  of  size  n  and  historic  proportion  P,  what  sample 
proportion  P  is  required  to  just  reach  the  5  percent 
significance  level? 


* 


1.  In  practice,  a  sampling  distribution  is  constructed  by  identifying 
the  appropriate  form  of  the  distribution  (e.g.,  normal  or  biriomial)  and 
then  estimating  its  parameters. 
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The  answer  is  computed  by  summing  terms  of  the  binomial  distribution 
from  zero  up  to  the  number  of  successes  m  where  the  5 -percent 
significance  level  is  reached.  In  equation  form,  this  expression  is: 

x-m 

probability  in  tail  -  1  -  S  - 77  F*  (1  -  P)n'X  <0.05  . 

x-0  *•  <'n  ‘  X)  ■ 

Reference  [B-2]  provides  extensive  tables  of  these  sums;  many 
statistics  textbooks  offer  restricted  tables  for  small  values  of  n,  but 
software  packages  for  statistics  probably  are  the  best  way  to  obtain 
exact  results.  Table  B-3  presents  successive  steps  in  the  summing 
process  for  four  different  levels  of  n  (the  number  of  boardings), 
demonstrating  how  the  probability  left  in  the  tail  declines  with 
successive  increases  in  the  number  of  successes  m.  The  asterisked 
entries  designate  the  points  where  the  probability  in  the  tail  just 
falls  below  0.05. 

For  example,  30  boardings  (one  month  of  operations  at  the  1988 
level  of  activity)  would  have  to  produce  a  boarding  success  rate  (Ps) 
over  13  percent,  more  than  three  times  the  baseline  rate.  Taking  a 
longer  period  to  accumulate  100  boardings  lowers  the  critical  rate  to 
8  percent,  still  twice  the  baseline  value.  Even  with  a  year's  worth  of 
data  (300  boardings)  in  the  new  sample,  the  boarding  success  rate  must 
grow  from  4  percent  to  6.3  percent,  an  increase  of  almost  58  percent. 
Extending  to  1,000  boardings,  51  successes  are  required  to  conclude 
there  is  a  difference;  the  ratio  to  the  historic  rate  is  1.275.  Thus, 
showing  that  a  27-percent  improvement  is  significant  requires  over  three 
years  of  operations  at  the  current  tempo.  Is  it  reasonable  to  expect 
that  drug  smugglers  would  not  change  their  methods  over  such  a  long 
time?  Would  U.S.  authorities  be  willing  to  wait  three  years  to  learn 
whether  an  operational  initiative  makes  a  difference? 


1.  As  sample  size  becomes  large  (greater  than  30),  the  normal 
distribution  is  a  good  approximation  to  the  binomial;  the  critical 
proportion  defining  a  5  percent  tail  may  be  computed  as 

Ps  -  P  +  1-645  ^jPCl  -  P). 

Put  another  way,  5  percent  of  the  area  under  the  normal  probability 
curve  is  contained  in  the  tail  beyond  1.645  standard  deviations  from  the 
mean. 
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Table  B-3.  Sample  proportions 
and  tall  probabilities 


Probability 


n 

m 

Pc  -  m/n 

^n  tail 

30 

2 

0.0667 

0.3388 

3 

0.1000 

0.1169 

4 

0.1333 

0.0316* 

5 

0.1667 

0.0063 

100 

6 

0.0600 

0.2116 

7 

0.0700 

0.1064 

8 

0.0800 

0.0475* 

9 

0.0900 

0.0190 

300 

17 

0.0567 

0.0969 

18 

0.0600 

0.0591 

19 

0.0633 

0.0343* 

20 

0.0667 

0.0190 

1,000 

49 

0.0490 

0.0881 

50 

0.0500 

0.0663 

51 

0.0510 

0.0491* 

52 

0.0520 

0.0357 

CONCLUSIONS  AND  COMMENTS 

Probability  and  statistics  may  be  used  to  draw  some  implications 
for  the  boarding  success  rate  as  an  MOE  for  drug  interdiction 
operations.  A  probability  model  allows  exploration  of  the  effects  from 
varying  sample  size  (number  of  boardings)  and  the  boarding  success 
rate.  An  extension  of  that  model  supports  an  investigation  of  the 
sample  size  necessary  to  conclude  that  an  observed  seizure  rate  is 
significantly  different  from  history.  Key  points  emerging  from  the 
analysis  are: 


•  Low  success  rates  naturally  mean  a  sizeable  fraction  of 
pulses  come  up  empty-handed,  as  observed  in  1988.  If 
maritime  forces  increase  seizures  markedly,  drug  smugglers 
are  likely  to  redirect  their  operations  to  other  avenues 
and  so  drive  seizure  rates  in  the  Caribbean  even  lower. 

It  will  be  a  major  challenge  of  leadership  to  sustain  an 
effective  drug  interdiction  operation  in  the  face  of 
infrequent  successes.  Both  crew  morale  and  the  question 
of  "what  are  we  getting  for  our  money"  will  be  difficult 
issues . 


•  Small  changes  in  success  rates  are  not  sufficient  to 
conclude  there  has  been  a  significant  change.  Huge 
margins  of  improvement  (doubling  and  tripling)  are 
necessary  to  be  confident  of  a  real  difference  when 
samples  represent  just  a  few  months  of  operations. 

Smaller  margins  might  be  believable  when  based  on  a  few 
years  of  operations,  but  stability  of  uncontrolled  factors 
is  a  serious  issue  over  any  long  time  span. 

Statistical  analysis  of  this  sort  can  give  commanders  and  other 
authorities  a  perspective  on  the  significance  of  performance  data. 
Ideally,  such  an  analysis  should  be  a  prelude  to  any  selection  of 
specific  MOEs  as  the  litmus  test  of  operational  success. 
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USING  POSTULATED  SCENARIOS 


Several  of  the  MOEs  explored  in  this  study  do  not  work  for  real 
operations  because  data  on  smuggler  activities  are  so  hard  to  obtain. 

In  theory,  perfect  intelligence  would  provide  the  means  to  tote  up 
departures  and  arrivals  of  drug  shipments  with  few  errors  or  oversights, 
but  information  collected  in  current  operations  falls  far  short  of  this 
ideal,  at  best  providing  a  sample  biased  toward  the  unsophisticated  end 
of  the  smuggler  spectrum.  Realistically,  even  the  smugglers  themselves 
probably  would  have  difficulty  collecting  comprehensive  and  reliable 
statistics,  assuming  they  had  a  reason  to  do  so.  Perhaps  dozens  of 
smuggling  operations  coexist  in  loose  association;  their  relationships 
vary  from  opportunistic  cooperation  to  murderous  competition.  Under 
these  circumstances,  drug  smugglers  probably  consider  hard  data  on  their 
activities  as  private  information  to  be  shared  sparingly  and  only  when 
there  is  a  good  reason.  Given  the  poor  prospects  of  ever  collecting 
such  data  on  a  wide  scale,  even  well  after  the  fact,  modeling  and 
simulation  methods  hold  some  appeal. 

USING  MODELS  TO  EXPLORE  IDEAS 

In  the  absence  of  ground  truth,  a  possible  approach  is  to  construct 
linked  models  of  the  opposing  interdiction  and  smuggling  campaigns  and 
then  investigate  their  joint  performance.  Similar  concepts  are  employed 
routinely  for  military  analyses  of  all  kinds.  Assessments  of  tank 
battles,  air  strikes,  submarine  engagements,  and  entire  theater 
campaigns  become  the  basis  for  decisions  on  budgets,  systems  acquisi¬ 
tion,  and  operational  plans.  Antidrug  interdiction  operations  seem  as 
amenable  to  such  treatment  as  these  other  subjects. 

Analytic  models  express  relationships  in  the  form  of  mathematical 
equations  whose  parameters  are  estimated  from  operational  data;  for 
example,  models  for  evaluating  radar  coverage  and  probability  of  detec¬ 
tion  against  specified  targets  are  well  developed.  Processes  not 
modeled  in  closed  form  can  be  investigated  via  computer  simulations. 

Once  constructed,  good  models  can  serve  many  purposes:  support  opera¬ 
tional  planning,  assess  effectiveness  of  operations,  and  explore  the 
consequences  of  changes  in  resources,  tactics,  or  any  other  "what  if" 
question  about  either  side. 

In  principle,  interdiction  forces  can  be  modeled  to  a  high  degree 
of  precision  because  all  the  resources  are  under  friendly  control,  but 
the  level  of  fidelity  likely  will  be  restrained  by  cost,  tolerance  for 
lag  time  in  producing  results,  and  the  operators'  patience  with  the 
data-collection  burden. 


C-l 


On  the  other  hand,  models  of  the  smuggling  side  depend  heavily  on 
how  well  intelligence  agencies  have  accumulated  an  understanding  of  the 
goals,  values,  methods,  and  other  factors  shaping  smuggler  operations. 
One  of  the  principal  challenges  is  to  model  all  elements  of  the 
smuggling  enterprise  as  a  whole  when  available  data  are  richest  in 
information  from  the  least  successful  players  (the  ones  that  were 
caught).  Even  here,  however,  such  modeling  holds  an  important  advantage 
over  methods  relying  solely  on  direct  estimation  from  collected  data: 
uncertainties  can  be  explored  through  alternative  descriptions  of  the 
enemy,  and  then  the  models  run  against  each  other  to  assess  whether  the 
apparent  differences  are  significant  to  the  performance  of  interdiction 
forces.  Such  a  process  offers  a  mechanism  to  call  attention  to  the  key 
issues,  focus  intelligence  activities  on  critical  indicators,  and  edu¬ 
cate  the  forces  on  their  own  weaknesses  that  smugglers  can  exploit. 

For  planning  purposes,  analyses  of  interdiction  forces  might  use 
programmatic  or  design  values  for  the  model  parameters.  After-the-fact 
evaluation  of  actual  operations  would  involve  running  the  models  with 
collected  data.  For  example,  planning  runs  would  take  as  inputs  the 
programmed  availability  of  sensors  and  interdiction  assets;  evaluation 
runs  would  use  the  actual  hours,  locations,  and  intensity  of  operations. 
Likewise,  the  smuggler  model  would  use  an  intelligence  forecast  for 
planning  and  then  incorporate  collected  data  for  evaluation. 

LIVE  EXERCISES  AND  TESTING  BY  SURROGATE  FORCES 

Whenever  one  becomes  heavily  dependent  on  models  for  descriptions 
of  fact,  it  is  wise  to  subject  the  results  to  organized  field  testing, 
the  more  realistic  and  demanding,  the  better.  In  the  case  of  drug 
interdiction,  an  obvious  device  is  to  assemble  a  team  of  skilled  people 
to  act  as  surrogates  for  the  smugglers.  They  might  study  the  organiza¬ 
tion  and  operations  of  interdiction  forces  to  identify  exploitable 
weaknesses  and  then  test  their  insights  with  live  infiltration,  perhaps 
using  vessels  and  aircraft  seized  from  smugglers.  Just  such  an  opera¬ 
tion  was  employed  in  Vietnam  to  evaluate  the  effectiveness  of  Market 
Time,  the  campaign  to  interdict  North  Vietnamese  maritime  smuggling  of 
arms  and  supplies  to  forces  operating  in  the  south.  A  U.S.  Navy  team 
studied  operations  for  cues  available  to  an  informed  smuggler  and  then, 
using  a  patrol  boat,  demonstrated  their  understanding  with  actual  pene¬ 
trations.  Accepted  wisdom  at  the  time  held  the  probability  of  detection 
for  a  penetrating  trawler  to  be  85  percent,  a  belief  demolished  when  the 
interdiction  forces  detected  only  one  of  the  surrogate's  nine  sorties. 
The  exercise  further  revealed  that  air  patrols  could  not  identify  infil¬ 
trators  at  night,  a  critical  fact  that  meant  a  sizeable  fraction  of 
expended  effort  was  wasted.  As  a  result,  operations  were  revised  radi¬ 
cally,  the  new  concepts  themselves  tested  in  further  penetration 


1.  See  CNA  Research  Contribution  280,  Market  Time  (U) ,  by  J.C.  Erdheim, 
Secret,  Sep  1975,  p.  42. 


exercises.  Such  testing  stands  to  produce  solid  data  on  the  actual 
capabilities  of  forces  because  investigators  can  instrument  both  sides 
of  the  event. 

SUMMARY 

This  concept  of  a  broadly  applicable  MOE,  linked  models,  and  live 
exercises  has  some  appeal  for  exploring  the  potential  performance  of 
changes  in  forces,  ta-tics,  and  operations;  however,  the  scheme  is  far 
from  perfect.  Scores  on  an  MOE  are  compiled  as  the  result  of  exercises 
or  modeling,  not  direct  measurements  from  actual  smuggling  operations. 
Consequently,  the  outputs  derive  their  credibility  from  the  candor, 
objectivity,  and  thoroughness  of  the  analytic  process,  rather  than  the 
fact  of  their  occurrence  as  actual  events .  There  should  always  remain  a 
concern  that  some  critical  aspect  of  the  smuggling  threat  is  not  ade¬ 
quately  represented  in  the  models  and  exercises;  at  the  very  least, 
smuggler  innovations  can  be  incorporated  only  after  they  become  well 
known,  a  fact  that  assures  important  time  lags  and  inevitable  revisions 
to  results  reported  earlier. 


